[ 760.370392] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 760.373616] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 760.525945] kselftest: Running tests in drivers/net
[ 989.681112] INFO: task python3:5411 blocked for more than 122 seconds.
[ 989.689312] Not tainted 7.1.0-rc6-xcxu-g6c6cff9f8fdc #1
[ 989.696856] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.706446] task:python3 state:D stack:25672 pid:5411 tgid:5411 ppid:5410 task_flags:0x400100 flags:0x00080000
[ 989.719654] Call Trace:
[ 989.723197]
[ 989.726347] __schedule+0xcc3/0x19c0
[ 989.731167] ? io_schedule_timeout+0x140/0x140
[ 989.736955] ? find_held_lock+0x2b/0x80
[ 989.742062] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.747956] schedule+0xd8/0x260
[ 989.752375] schedule_preempt_disabled+0x18/0x30
[ 989.758354] __mutex_lock+0x6af/0x2010
[ 989.763359] ? threaded_store+0x132/0x220
[ 989.768659] ? ww_mutex_lock+0x160/0x160
[ 989.773860] ? selinux_inode_free_security+0x330/0x330
[ 989.780431] ? cap_capable+0x1d6/0x3e0
[ 989.785439] ? threaded_store+0x132/0x220
[ 989.790733] threaded_store+0x132/0x220
[ 989.795834] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.801132] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.806433] ? sysfs_kf_write+0x65/0x160
[ 989.811636] ? sysfs_file_ops+0x140/0x140
[ 989.816935] kernfs_fop_write_iter+0x335/0x510
[ 989.822720] ? kernfs_ops+0xf0/0xf0
[ 989.827432] new_sync_write+0x333/0x750
[ 989.832539] ? __lock_acquire+0x508/0xc10
[ 989.837835] ? new_sync_read+0x740/0x740
[ 989.843035] ? lock_acquire.part.0+0xbc/0x260
[ 989.848723] ? ksys_write+0x116/0x250
[ 989.853633] vfs_write+0x657/0x9b0
[ 989.858247] ksys_write+0x116/0x250
[ 989.862958] ? __ia32_sys_read+0xc0/0xc0
[ 989.868158] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.873749] ? rcu_is_watching+0x15/0xd0
[ 989.878956] do_syscall_64+0xf3/0x650
[ 989.883863] ? trace_hardirqs_off+0xd/0x30
[ 989.889261] ? exc_page_fault+0xda/0xf0
[ 989.894353] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.900820] RIP: 0033:0x7ff5e417664e
[ 989.905630] RSP: 002b:00007fffe96931d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.914931] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff5e417664e
[ 989.923738] RDX: 0000000000000001 RSI: 000055ecb444abf0 RDI: 0000000000000007
[ 989.932544] RBP: 00007fffe96931e0 R08: 0000000000000000 R09: 0000000000000000
[ 989.941356] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff5e3fef390
[ 989.950162] R13: 000055ecb444abf0 R14: 0000000000000007 R15: 00007ff5e3f85120
[ 989.958984]
[ 989.962294] INFO: task python3:5411 is blocked on a mutex likely owned by task python3:3421.
[ 989.972568] task:python3 state:D stack:25672 pid:3421 tgid:3421 ppid:3420 task_flags:0x400100 flags:0x00080802
[ 989.985773] Call Trace:
[ 989.989311]
[ 989.992464] __schedule+0xcc3/0x19c0
[ 989.997278] ? io_schedule_timeout+0x140/0x140
[ 990.003063] ? find_held_lock+0x2b/0x80
[ 990.008166] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.014344] schedule+0xd8/0x260
[ 990.018763] schedule_timeout+0x13f/0x290
[ 990.024059] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.030429] ? trace_contention_end+0xb3/0x1a0
[ 990.036217] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.041909] ? ww_mutex_lock+0x160/0x160
[ 990.047111] msleep+0x1b/0x30
[ 990.051238] napi_set_threaded+0x3a5/0x440
[ 990.056633] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.062126] netif_set_threaded+0x257/0x400
[ 990.067611] threaded_store+0x1b8/0x220
[ 990.072716] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.078014] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.083309] ? sysfs_kf_write+0x65/0x160
[ 990.088509] ? sysfs_file_ops+0x140/0x140
[ 990.093797] kernfs_fop_write_iter+0x335/0x510
[ 990.099584] ? kernfs_ops+0xf0/0xf0
[ 990.104295] new_sync_write+0x333/0x750
[ 990.109398] ? __lock_acquire+0x508/0xc10
[ 990.114693] ? new_sync_read+0x740/0x740
[ 990.119895] ? lock_acquire.part.0+0xbc/0x260
[ 990.125581] ? ksys_write+0x116/0x250
[ 990.130489] vfs_write+0x657/0x9b0
[ 990.135108] ksys_write+0x116/0x250
[ 990.139822] ? __ia32_sys_read+0xc0/0xc0
[ 990.145019] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.150607] ? rcu_is_watching+0x15/0xd0
[ 990.155810] do_syscall_64+0xf3/0x650
[ 990.160718] ? trace_hardirqs_off+0xd/0x30
[ 990.166113] ? exc_page_fault+0xda/0xf0
[ 990.171208] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.177673] RIP: 0033:0x7f82564ab64e
[ 990.182482] RSP: 002b:00007ffceb2921f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.191779] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f82564ab64e
[ 990.200589] RDX: 0000000000000001 RSI: 00005616c4ee5bf0 RDI: 0000000000000007
[ 990.209399] RBP: 00007ffceb292200 R08: 0000000000000000 R09: 0000000000000000
[ 990.218209] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8256324390
[ 990.227018] R13: 00005616c4ee5bf0 R14: 0000000000000007 R15: 00007f82562b9120
[ 990.235832]
[ 990.239096] \x0aShowing all locks held in the system:
[ 990.247638] 1 lock held by khungtaskd/185:
[ 990.253023] #0: ffffffff93cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.264511] 4 locks held by python3/3421:
[ 990.269806] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.279698] #1: ff110001405e1480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.290660] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.301819] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.311921] 4 locks held by python3/5411:
[ 990.317218] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.327117] #1: ff110001af386880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.338097] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.349260] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.359350]
[ 990.361807] =============================================\x0a
[ 1112.562168] INFO: task python3:5411 blocked for more than 245 seconds.
[ 1112.570315] Not tainted 7.1.0-rc6-xcxu-g6c6cff9f8fdc #1
[ 1112.577566] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.587155] task:python3 state:D stack:25672 pid:5411 tgid:5411 ppid:5410 task_flags:0x400100 flags:0x00080000
[ 1112.600361] Call Trace:
[ 1112.603903]
[ 1112.607056] __schedule+0xcc3/0x19c0
[ 1112.611871] ? io_schedule_timeout+0x140/0x140
[ 1112.617658] ? find_held_lock+0x2b/0x80
[ 1112.622763] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.628647] schedule+0xd8/0x260
[ 1112.633065] schedule_preempt_disabled+0x18/0x30
[ 1112.639043] __mutex_lock+0x6af/0x2010
[ 1112.644048] ? threaded_store+0x132/0x220
[ 1112.649338] ? ww_mutex_lock+0x160/0x160
[ 1112.654539] ? selinux_inode_free_security+0x330/0x330
[ 1112.661108] ? cap_capable+0x1d6/0x3e0
[ 1112.666115] ? threaded_store+0x132/0x220
[ 1112.671403] threaded_store+0x132/0x220
[ 1112.676505] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.681804] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.687106] ? sysfs_kf_write+0x65/0x160
[ 1112.692305] ? sysfs_file_ops+0x140/0x140
[ 1112.697600] kernfs_fop_write_iter+0x335/0x510
[ 1112.703385] ? kernfs_ops+0xf0/0xf0
[ 1112.708098] new_sync_write+0x333/0x750
[ 1112.713201] ? __lock_acquire+0x508/0xc10
[ 1112.718497] ? new_sync_read+0x740/0x740
[ 1112.723698] ? lock_acquire.part.0+0xbc/0x260
[ 1112.729376] ? ksys_write+0x116/0x250
[ 1112.734276] vfs_write+0x657/0x9b0
[ 1112.738891] ksys_write+0x116/0x250
[ 1112.743601] ? __ia32_sys_read+0xc0/0xc0
[ 1112.748801] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.754390] ? rcu_is_watching+0x15/0xd0
[ 1112.759591] do_syscall_64+0xf3/0x650
[ 1112.764497] ? trace_hardirqs_off+0xd/0x30
[ 1112.769891] ? exc_page_fault+0xda/0xf0
[ 1112.774994] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.781463] RIP: 0033:0x7ff5e417664e
[ 1112.786274] RSP: 002b:00007fffe96931d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.795571] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff5e417664e
[ 1112.804378] RDX: 0000000000000001 RSI: 000055ecb444abf0 RDI: 0000000000000007
[ 1112.813184] RBP: 00007fffe96931e0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.821994] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff5e3fef390
[ 1112.830801] R13: 000055ecb444abf0 R14: 0000000000000007 R15: 00007ff5e3f85120
[ 1112.839613]
[ 1112.842910] INFO: task python3:5411 is blocked on a mutex likely owned by task python3:3421.
[ 1112.853182] task:python3 state:D stack:25672 pid:3421 tgid:3421 ppid:3420 task_flags:0x400100 flags:0x00080802
[ 1112.866384] Call Trace:
[ 1112.869922]
[ 1112.873076] __schedule+0xcc3/0x19c0
[ 1112.877885] ? io_schedule_timeout+0x140/0x140
[ 1112.883671] ? find_held_lock+0x2b/0x80
[ 1112.888774] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.894952] schedule+0xd8/0x260
[ 1112.899371] schedule_timeout+0x13f/0x290
[ 1112.904669] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.911039] ? trace_contention_end+0xb3/0x1a0
[ 1112.916824] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.922507] ? ww_mutex_lock+0x160/0x160
[ 1112.927705] msleep+0x1b/0x30
[ 1112.931833] napi_set_threaded+0x3a5/0x440
[ 1112.937229] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.942722] netif_set_threaded+0x257/0x400
[ 1112.948218] threaded_store+0x1b8/0x220
[ 1112.953320] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.958616] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.963911] ? sysfs_kf_write+0x65/0x160
[ 1112.969113] ? sysfs_file_ops+0x140/0x140
[ 1112.974407] kernfs_fop_write_iter+0x335/0x510
[ 1112.980191] ? kernfs_ops+0xf0/0xf0
[ 1112.984903] new_sync_write+0x333/0x750
[ 1112.990014] ? __lock_acquire+0x508/0xc10
[ 1112.995300] ? new_sync_read+0x740/0x740
[ 1113.000498] ? lock_acquire.part.0+0xbc/0x260
[ 1113.006186] ? ksys_write+0x116/0x250
[ 1113.011087] vfs_write+0x657/0x9b0
[ 1113.015702] ksys_write+0x116/0x250
[ 1113.020407] ? __ia32_sys_read+0xc0/0xc0
[ 1113.025606] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.031195] ? rcu_is_watching+0x15/0xd0
[ 1113.036396] do_syscall_64+0xf3/0x650
[ 1113.041304] ? trace_hardirqs_off+0xd/0x30
[ 1113.046699] ? exc_page_fault+0xda/0xf0
[ 1113.051803] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.058268] RIP: 0033:0x7f82564ab64e
[ 1113.063079] RSP: 002b:00007ffceb2921f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.072375] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f82564ab64e
[ 1113.081182] RDX: 0000000000000001 RSI: 00005616c4ee5bf0 RDI: 0000000000000007
[ 1113.089989] RBP: 00007ffceb292200 R08: 0000000000000000 R09: 0000000000000000
[ 1113.098788] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8256324390
[ 1113.107597] R13: 00005616c4ee5bf0 R14: 0000000000000007 R15: 00007f82562b9120
[ 1113.116409]
[ 1113.119672] \x0aShowing all locks held in the system:
[ 1113.128210] 1 lock held by khungtaskd/185:
[ 1113.133604] #0: ffffffff93cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.145075] 3 locks held by systemd-journal/806:
[ 1113.151057] #0: ff1100180e3374e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.160939] #1: ffffffff93f02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.173757] #2: ff110001a5179a40 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1113.185315] 4 locks held by python3/3421:
[ 1113.190612] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.200498] #1: ff110001405e1480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.211461] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.222616] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.232712] 4 locks held by python3/5411:
[ 1113.238015] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.247902] #1: ff110001af386880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.258863] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.270024] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.280113]
[ 1113.282581] =============================================\x0a
[ 1235.443313] INFO: task python3:5411 blocked for more than 368 seconds.
[ 1235.451461] Not tainted 7.1.0-rc6-xcxu-g6c6cff9f8fdc #1
[ 1235.458713] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.468305] task:python3 state:D stack:25672 pid:5411 tgid:5411 ppid:5410 task_flags:0x400100 flags:0x00080002
[ 1235.481512] Call Trace:
[ 1235.485054]
[ 1235.488203] __schedule+0xcc3/0x19c0
[ 1235.493019] ? io_schedule_timeout+0x140/0x140
[ 1235.498807] ? find_held_lock+0x2b/0x80
[ 1235.503911] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.509796] schedule+0xd8/0x260
[ 1235.514218] schedule_preempt_disabled+0x18/0x30
[ 1235.520197] __mutex_lock+0x6af/0x2010
[ 1235.525195] ? threaded_store+0x132/0x220
[ 1235.530494] ? ww_mutex_lock+0x160/0x160
[ 1235.535694] ? selinux_inode_free_security+0x330/0x330
[ 1235.542267] ? cap_capable+0x1d6/0x3e0
[ 1235.547274] ? threaded_store+0x132/0x220
[ 1235.552572] threaded_store+0x132/0x220
[ 1235.557673] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.562972] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.568272] ? sysfs_kf_write+0x65/0x160
[ 1235.573464] ? sysfs_file_ops+0x140/0x140
[ 1235.578762] kernfs_fop_write_iter+0x335/0x510
[ 1235.584547] ? kernfs_ops+0xf0/0xf0
[ 1235.589260] new_sync_write+0x333/0x750
[ 1235.594361] ? __lock_acquire+0x508/0xc10
[ 1235.599660] ? new_sync_read+0x740/0x740
[ 1235.604859] ? lock_acquire.part.0+0xbc/0x260
[ 1235.610546] ? ksys_write+0x116/0x250
[ 1235.615457] vfs_write+0x657/0x9b0
[ 1235.620075] ksys_write+0x116/0x250
[ 1235.624788] ? __ia32_sys_read+0xc0/0xc0
[ 1235.629987] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.635579] ? rcu_is_watching+0x15/0xd0
[ 1235.640781] do_syscall_64+0xf3/0x650
[ 1235.645686] ? trace_hardirqs_off+0xd/0x30
[ 1235.651083] ? exc_page_fault+0xda/0xf0
[ 1235.656176] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.662643] RIP: 0033:0x7ff5e417664e
[ 1235.667445] RSP: 002b:00007fffe96931d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.676736] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff5e417664e
[ 1235.685544] RDX: 0000000000000001 RSI: 000055ecb444abf0 RDI: 0000000000000007
[ 1235.694352] RBP: 00007fffe96931e0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.703165] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff5e3fef390
[ 1235.711972] R13: 000055ecb444abf0 R14: 0000000000000007 R15: 00007ff5e3f85120
[ 1235.720778]
[ 1235.724048] INFO: task python3:5411 is blocked on a mutex likely owned by task python3:3421.
[ 1235.734321] task:python3 state:D stack:25672 pid:3421 tgid:3421 ppid:3420 task_flags:0x400100 flags:0x00080802
[ 1235.747524] Call Trace:
[ 1235.751064]
[ 1235.754214] __schedule+0xcc3/0x19c0
[ 1235.759024] ? io_schedule_timeout+0x140/0x140
[ 1235.764801] ? find_held_lock+0x2b/0x80
[ 1235.769903] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.776083] schedule+0xd8/0x260
[ 1235.780503] schedule_timeout+0x13f/0x290
[ 1235.785792] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.792160] ? trace_contention_end+0xb3/0x1a0
[ 1235.797945] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.803626] ? ww_mutex_lock+0x160/0x160
[ 1235.808828] msleep+0x1b/0x30
[ 1235.812956] napi_set_threaded+0x3a5/0x440
[ 1235.818352] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.823839] netif_set_threaded+0x257/0x400
[ 1235.829331] threaded_store+0x1b8/0x220
[ 1235.834433] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.839720] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.845016] ? sysfs_kf_write+0x65/0x160
[ 1235.850215] ? sysfs_file_ops+0x140/0x140
[ 1235.855511] kernfs_fop_write_iter+0x335/0x510
[ 1235.861296] ? kernfs_ops+0xf0/0xf0
[ 1235.865997] new_sync_write+0x333/0x750
[ 1235.871098] ? __lock_acquire+0x508/0xc10
[ 1235.876395] ? new_sync_read+0x740/0x740
[ 1235.881586] ? lock_acquire.part.0+0xbc/0x260
[ 1235.887270] ? ksys_write+0x116/0x250
[ 1235.892182] vfs_write+0x657/0x9b0
[ 1235.896799] ksys_write+0x116/0x250
[ 1235.901510] ? __ia32_sys_read+0xc0/0xc0
[ 1235.906703] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.912292] ? rcu_is_watching+0x15/0xd0
[ 1235.917494] do_syscall_64+0xf3/0x650
[ 1235.922399] ? trace_hardirqs_off+0xd/0x30
[ 1235.927792] ? exc_page_fault+0xda/0xf0
[ 1235.932893] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.939360] RIP: 0033:0x7f82564ab64e
[ 1235.944163] RSP: 002b:00007ffceb2921f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.953461] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f82564ab64e
[ 1235.962269] RDX: 0000000000000001 RSI: 00005616c4ee5bf0 RDI: 0000000000000007
[ 1235.971079] RBP: 00007ffceb292200 R08: 0000000000000000 R09: 0000000000000000
[ 1235.979889] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8256324390
[ 1235.988695] R13: 00005616c4ee5bf0 R14: 0000000000000007 R15: 00007f82562b9120
[ 1235.997510]
[ 1236.000776] \x0aShowing all locks held in the system:
[ 1236.009314] 1 lock held by khungtaskd/185:
[ 1236.014706] #0: ffffffff93cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.026193] 4 locks held by python3/3421:
[ 1236.031488] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.041379] #1: ff110001405e1480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.052340] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.063495] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.073587] 4 locks held by python3/5411:
[ 1236.078883] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.088772] #1: ff110001af386880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.099733] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.110893] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.120986]
[ 1236.123451] =============================================\x0a
[ 1264.675214] block nvme0n1: No UUID available providing old NGUID
SUBSYSTEM=block
DEVICE=b259:0
[ 1346.036282] kworker/u130:5 (761) used greatest stack depth: 21936 bytes left
[ 1358.324462] INFO: task python3:5411 blocked for more than 491 seconds.
[ 1358.332610] Not tainted 7.1.0-rc6-xcxu-g6c6cff9f8fdc #1
[ 1358.339861] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.349450] task:python3 state:D stack:25672 pid:5411 tgid:5411 ppid:5410 task_flags:0x400100 flags:0x00080002
[ 1358.362656] Call Trace:
[ 1358.366198]
[ 1358.369351] __schedule+0xcc3/0x19c0
[ 1358.374170] ? io_schedule_timeout+0x140/0x140
[ 1358.379959] ? find_held_lock+0x2b/0x80
[ 1358.385066] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.390951] schedule+0xd8/0x260
[ 1358.395370] schedule_preempt_disabled+0x18/0x30
[ 1358.401348] __mutex_lock+0x6af/0x2010
[ 1358.406351] ? threaded_store+0x132/0x220
[ 1358.411651] ? ww_mutex_lock+0x160/0x160
[ 1358.416851] ? selinux_inode_free_security+0x330/0x330
[ 1358.423421] ? cap_capable+0x1d6/0x3e0
[ 1358.428431] ? threaded_store+0x132/0x220
[ 1358.433729] threaded_store+0x132/0x220
[ 1358.438820] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.444118] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.449417] ? sysfs_kf_write+0x65/0x160
[ 1358.454613] ? sysfs_file_ops+0x140/0x140
[ 1358.459911] kernfs_fop_write_iter+0x335/0x510
[ 1358.465694] ? kernfs_ops+0xf0/0xf0
[ 1358.470406] new_sync_write+0x333/0x750
[ 1358.475512] ? __lock_acquire+0x508/0xc10
[ 1358.480810] ? new_sync_read+0x740/0x740
[ 1358.486013] ? lock_acquire.part.0+0xbc/0x260
[ 1358.491701] ? ksys_write+0x116/0x250
[ 1358.496610] vfs_write+0x657/0x9b0
[ 1358.501228] ksys_write+0x116/0x250
[ 1358.505938] ? __ia32_sys_read+0xc0/0xc0
[ 1358.511140] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.516730] ? rcu_is_watching+0x15/0xd0
[ 1358.521931] do_syscall_64+0xf3/0x650
[ 1358.526839] ? trace_hardirqs_off+0xd/0x30
[ 1358.532235] ? exc_page_fault+0xda/0xf0
[ 1358.537338] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.543808] RIP: 0033:0x7ff5e417664e
[ 1358.548620] RSP: 002b:00007fffe96931d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.557919] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff5e417664e
[ 1358.566729] RDX: 0000000000000001 RSI: 000055ecb444abf0 RDI: 0000000000000007
[ 1358.575539] RBP: 00007fffe96931e0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.584349] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff5e3fef390
[ 1358.593156] R13: 000055ecb444abf0 R14: 0000000000000007 R15: 00007ff5e3f85120
[ 1358.601969]
[ 1358.605254] INFO: task python3:5411 is blocked on a mutex likely owned by task python3:3421.
[ 1358.615526] task:python3 state:D stack:25672 pid:3421 tgid:3421 ppid:3420 task_flags:0x400100 flags:0x00080802
[ 1358.628730] Call Trace:
[ 1358.632269]
[ 1358.635422] __schedule+0xcc3/0x19c0
[ 1358.640234] ? io_schedule_timeout+0x140/0x140
[ 1358.646021] ? find_held_lock+0x2b/0x80
[ 1358.651126] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.657308] schedule+0xd8/0x260
[ 1358.661727] schedule_timeout+0x13f/0x290
[ 1358.667023] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.673391] ? trace_contention_end+0xb3/0x1a0
[ 1358.679178] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.684869] ? ww_mutex_lock+0x160/0x160
[ 1358.690070] msleep+0x1b/0x30
[ 1358.694194] napi_set_threaded+0x3a5/0x440
[ 1358.699593] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.705086] netif_set_threaded+0x257/0x400
[ 1358.710570] threaded_store+0x1b8/0x220
[ 1358.715662] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.720959] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.726254] ? sysfs_kf_write+0x65/0x160
[ 1358.731455] ? sysfs_file_ops+0x140/0x140
[ 1358.736753] kernfs_fop_write_iter+0x335/0x510
[ 1358.742531] ? kernfs_ops+0xf0/0xf0
[ 1358.747243] new_sync_write+0x333/0x750
[ 1358.752346] ? __lock_acquire+0x508/0xc10
[ 1358.757633] ? new_sync_read+0x740/0x740
[ 1358.762834] ? lock_acquire.part.0+0xbc/0x260
[ 1358.768521] ? ksys_write+0x116/0x250
[ 1358.773430] vfs_write+0x657/0x9b0
[ 1358.778044] ksys_write+0x116/0x250
[ 1358.782757] ? __ia32_sys_read+0xc0/0xc0
[ 1358.787955] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.793547] ? rcu_is_watching+0x15/0xd0
[ 1358.798746] do_syscall_64+0xf3/0x650
[ 1358.803651] ? trace_hardirqs_off+0xd/0x30
[ 1358.809045] ? exc_page_fault+0xda/0xf0
[ 1358.814147] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.820615] RIP: 0033:0x7f82564ab64e
[ 1358.825425] RSP: 002b:00007ffceb2921f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.834722] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f82564ab64e
[ 1358.843529] RDX: 0000000000000001 RSI: 00005616c4ee5bf0 RDI: 0000000000000007
[ 1358.852338] RBP: 00007ffceb292200 R08: 0000000000000000 R09: 0000000000000000
[ 1358.861146] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8256324390
[ 1358.869954] R13: 00005616c4ee5bf0 R14: 0000000000000007 R15: 00007f82562b9120
[ 1358.878768]
[ 1358.882037] \x0aShowing all locks held in the system:
[ 1358.890576] 1 lock held by khungtaskd/185:
[ 1358.895968] #0: ffffffff93cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.907450] 4 locks held by python3/3421:
[ 1358.912744] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.922636] #1: ff110001405e1480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.933596] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.944753] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.954848] 4 locks held by python3/5411:
[ 1358.960144] #0: ff1100012f8cc3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.970033] #1: ff110001af386880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.980997] #2: ff110001222a0b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.992154] #3: ff110001222e9db8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.002247]
[ 1359.004713] =============================================\x0a