[ 768.291260] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 768.294646] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 768.422166] kselftest: Running tests in drivers/net
[ 989.683850] INFO: task python3:5517 blocked for more than 122 seconds.
[ 989.692081] Not tainted 7.1.0-rc6-cysg-g81483f60bc8a #1
[ 989.699332] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.709635] task:python3 state:D stack:25672 pid:5517 tgid:5517 ppid:5516 task_flags:0x400100 flags:0x00080000
[ 989.722846] Call Trace:
[ 989.726382]
[ 989.729530] __schedule+0xcc3/0x19c0
[ 989.734349] ? io_schedule_timeout+0x140/0x140
[ 989.740133] ? find_held_lock+0x2b/0x80
[ 989.745236] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.751120] schedule+0xd8/0x260
[ 989.755537] schedule_preempt_disabled+0x18/0x30
[ 989.761513] __mutex_lock+0x6af/0x2010
[ 989.766520] ? threaded_store+0x132/0x220
[ 989.771820] ? ww_mutex_lock+0x160/0x160
[ 989.777018] ? selinux_inode_free_security+0x330/0x330
[ 989.783580] ? cap_capable+0x1d6/0x3e0
[ 989.788587] ? threaded_store+0x132/0x220
[ 989.793882] threaded_store+0x132/0x220
[ 989.798984] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.804282] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.809578] ? sysfs_kf_write+0x65/0x160
[ 989.814780] ? sysfs_file_ops+0x140/0x140
[ 989.820075] kernfs_fop_write_iter+0x335/0x510
[ 989.825858] ? kernfs_ops+0xf0/0xf0
[ 989.830567] new_sync_write+0x333/0x750
[ 989.835670] ? __lock_acquire+0x508/0xc10
[ 989.840967] ? new_sync_read+0x740/0x740
[ 989.846168] ? lock_acquire.part.0+0xbc/0x260
[ 989.851854] ? ksys_write+0x116/0x250
[ 989.856762] vfs_write+0x657/0x9b0
[ 989.861376] ksys_write+0x116/0x250
[ 989.866085] ? __ia32_sys_read+0xc0/0xc0
[ 989.871281] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.876867] ? rcu_is_watching+0x15/0xd0
[ 989.882067] do_syscall_64+0xf3/0x650
[ 989.886974] ? trace_hardirqs_off+0xd/0x30
[ 989.892367] ? exc_page_fault+0xda/0xf0
[ 989.897468] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.903926] RIP: 0033:0x7fb75acb364e
[ 989.908733] RSP: 002b:00007ffeddbc19b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.918029] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb75acb364e
[ 989.926837] RDX: 0000000000000001 RSI: 000055d5e234abf0 RDI: 0000000000000007
[ 989.935644] RBP: 00007ffeddbc19c0 R08: 0000000000000000 R09: 0000000000000000
[ 989.944451] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb75ab2c390
[ 989.953329] R13: 000055d5e234abf0 R14: 0000000000000007 R15: 00007fb75aac1120
[ 989.962150]
[ 989.965490] INFO: task python3:5517 is blocked on a mutex likely owned by task python3:3478.
[ 989.975759] task:python3 state:D stack:25672 pid:3478 tgid:3478 ppid:3477 task_flags:0x400100 flags:0x00080802
[ 989.988960] Call Trace:
[ 989.992495]
[ 989.995729] __schedule+0xcc3/0x19c0
[ 990.000541] ? io_schedule_timeout+0x140/0x140
[ 990.006325] ? find_held_lock+0x2b/0x80
[ 990.011425] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.017602] schedule+0xd8/0x260
[ 990.022021] schedule_timeout+0x13f/0x290
[ 990.027318] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.033689] ? trace_contention_end+0xb3/0x1a0
[ 990.039472] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.045159] ? ww_mutex_lock+0x160/0x160
[ 990.050361] msleep+0x1b/0x30
[ 990.054487] napi_set_threaded+0x3a5/0x440
[ 990.059881] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.065375] netif_set_threaded+0x257/0x400
[ 990.070868] threaded_store+0x1b8/0x220
[ 990.075967] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.081265] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.086561] ? sysfs_kf_write+0x65/0x160
[ 990.091760] ? sysfs_file_ops+0x140/0x140
[ 990.097056] kernfs_fop_write_iter+0x335/0x510
[ 990.102841] ? kernfs_ops+0xf0/0xf0
[ 990.107544] new_sync_write+0x333/0x750
[ 990.112646] ? __lock_acquire+0x508/0xc10
[ 990.117940] ? new_sync_read+0x740/0x740
[ 990.123141] ? lock_acquire.part.0+0xbc/0x260
[ 990.128825] ? ksys_write+0x116/0x250
[ 990.133733] vfs_write+0x657/0x9b0
[ 990.138345] ksys_write+0x116/0x250
[ 990.143055] ? __ia32_sys_read+0xc0/0xc0
[ 990.148254] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.153841] ? rcu_is_watching+0x15/0xd0
[ 990.159040] do_syscall_64+0xf3/0x650
[ 990.163943] ? irq_exit_rcu+0x1a/0x30
[ 990.168849] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.175313] RIP: 0033:0x7f76dc6a664e
[ 990.180119] RSP: 002b:00007ffd80fde200 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.189415] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f76dc6a664e
[ 990.198222] RDX: 0000000000000001 RSI: 000055b8a1c3abf0 RDI: 0000000000000007
[ 990.207028] RBP: 00007ffd80fde210 R08: 0000000000000000 R09: 0000000000000000
[ 990.215835] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f76dc51f390
[ 990.224639] R13: 000055b8a1c3abf0 R14: 0000000000000007 R15: 00007f76dc4b5120
[ 990.233450]
[ 990.236710] \x0aShowing all locks held in the system:
[ 990.245251] 1 lock held by khungtaskd/185:
[ 990.250643] #0: ffffffffb2cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.262119] 3 locks held by systemd-journal/805:
[ 990.268095] #0: ff1100180e1b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.277981] #1: ffffffffb2f02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.290797] #2: ff110001a57fe020 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.302353] 4 locks held by python3/3478:
[ 990.307640] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.317601] #1: ff110001c3516e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.328569] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.339740] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.349856] 4 locks held by python3/5517:
[ 990.355149] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.365035] #1: ff11000180a9c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.375998] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.387154] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.397297]
[ 990.399828] =============================================\x0a
[ 1112.564962] INFO: task python3:5517 blocked for more than 245 seconds.
[ 1112.573113] Not tainted 7.1.0-rc6-cysg-g81483f60bc8a #1
[ 1112.580363] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.589950] task:python3 state:D stack:25672 pid:5517 tgid:5517 ppid:5516 task_flags:0x400100 flags:0x00080000
[ 1112.603155] Call Trace:
[ 1112.606695]
[ 1112.609847] __schedule+0xcc3/0x19c0
[ 1112.614661] ? io_schedule_timeout+0x140/0x140
[ 1112.620447] ? find_held_lock+0x2b/0x80
[ 1112.625549] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.631435] schedule+0xd8/0x260
[ 1112.635851] schedule_preempt_disabled+0x18/0x30
[ 1112.641829] __mutex_lock+0x6af/0x2010
[ 1112.646832] ? threaded_store+0x132/0x220
[ 1112.652130] ? ww_mutex_lock+0x160/0x160
[ 1112.657326] ? selinux_inode_free_security+0x330/0x330
[ 1112.663897] ? cap_capable+0x1d6/0x3e0
[ 1112.668905] ? threaded_store+0x132/0x220
[ 1112.674205] threaded_store+0x132/0x220
[ 1112.679306] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.684601] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.689897] ? sysfs_kf_write+0x65/0x160
[ 1112.695098] ? sysfs_file_ops+0x140/0x140
[ 1112.700390] kernfs_fop_write_iter+0x335/0x510
[ 1112.706173] ? kernfs_ops+0xf0/0xf0
[ 1112.710883] new_sync_write+0x333/0x750
[ 1112.715988] ? __lock_acquire+0x508/0xc10
[ 1112.721283] ? new_sync_read+0x740/0x740
[ 1112.726483] ? lock_acquire.part.0+0xbc/0x260
[ 1112.732169] ? ksys_write+0x116/0x250
[ 1112.737077] vfs_write+0x657/0x9b0
[ 1112.741681] ksys_write+0x116/0x250
[ 1112.746393] ? __ia32_sys_read+0xc0/0xc0
[ 1112.751581] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.757167] ? rcu_is_watching+0x15/0xd0
[ 1112.762369] do_syscall_64+0xf3/0x650
[ 1112.767275] ? trace_hardirqs_off+0xd/0x30
[ 1112.772670] ? exc_page_fault+0xda/0xf0
[ 1112.777773] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.784240] RIP: 0033:0x7fb75acb364e
[ 1112.789047] RSP: 002b:00007ffeddbc19b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.798343] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb75acb364e
[ 1112.807150] RDX: 0000000000000001 RSI: 000055d5e234abf0 RDI: 0000000000000007
[ 1112.815956] RBP: 00007ffeddbc19c0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.824762] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb75ab2c390
[ 1112.833570] R13: 000055d5e234abf0 R14: 0000000000000007 R15: 00007fb75aac1120
[ 1112.842372]
[ 1112.845658] INFO: task python3:5517 is blocked on a mutex likely owned by task python3:3478.
[ 1112.855930] task:python3 state:D stack:25672 pid:3478 tgid:3478 ppid:3477 task_flags:0x400100 flags:0x00080802
[ 1112.869130] Call Trace:
[ 1112.872668]
[ 1112.875816] __schedule+0xcc3/0x19c0
[ 1112.880625] ? io_schedule_timeout+0x140/0x140
[ 1112.886411] ? find_held_lock+0x2b/0x80
[ 1112.891511] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.897687] schedule+0xd8/0x260
[ 1112.902097] schedule_timeout+0x13f/0x290
[ 1112.907392] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.913760] ? trace_contention_end+0xb3/0x1a0
[ 1112.919543] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.925233] ? ww_mutex_lock+0x160/0x160
[ 1112.930430] msleep+0x1b/0x30
[ 1112.934552] napi_set_threaded+0x3a5/0x440
[ 1112.939949] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.945434] netif_set_threaded+0x257/0x400
[ 1112.950928] threaded_store+0x1b8/0x220
[ 1112.956030] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.961326] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.966623] ? sysfs_kf_write+0x65/0x160
[ 1112.971820] ? sysfs_file_ops+0x140/0x140
[ 1112.977115] kernfs_fop_write_iter+0x335/0x510
[ 1112.982900] ? kernfs_ops+0xf0/0xf0
[ 1112.987609] new_sync_write+0x333/0x750
[ 1112.992709] ? __lock_acquire+0x508/0xc10
[ 1112.998006] ? new_sync_read+0x740/0x740
[ 1113.003205] ? lock_acquire.part.0+0xbc/0x260
[ 1113.008889] ? ksys_write+0x116/0x250
[ 1113.013802] vfs_write+0x657/0x9b0
[ 1113.018418] ksys_write+0x116/0x250
[ 1113.023129] ? __ia32_sys_read+0xc0/0xc0
[ 1113.028329] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.033918] ? rcu_is_watching+0x15/0xd0
[ 1113.039118] do_syscall_64+0xf3/0x650
[ 1113.044025] ? irq_exit_rcu+0x1a/0x30
[ 1113.048922] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.055388] RIP: 0033:0x7f76dc6a664e
[ 1113.060195] RSP: 002b:00007ffd80fde200 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.069491] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f76dc6a664e
[ 1113.078291] RDX: 0000000000000001 RSI: 000055b8a1c3abf0 RDI: 0000000000000007
[ 1113.087098] RBP: 00007ffd80fde210 R08: 0000000000000000 R09: 0000000000000000
[ 1113.095898] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f76dc51f390
[ 1113.104705] R13: 000055b8a1c3abf0 R14: 0000000000000007 R15: 00007f76dc4b5120
[ 1113.113510]
[ 1113.116772] \x0aShowing all locks held in the system:
[ 1113.125311] 1 lock held by khungtaskd/185:
[ 1113.130704] #0: ffffffffb2cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.142175] 3 locks held by systemd-journal/805:
[ 1113.148154] #0: ff1100180df374e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.158040] #1: ffffffffb2f02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1113.170271] #2: ff110001a57fe020 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1113.181829] 4 locks held by python3/3478:
[ 1113.187123] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.197010] #1: ff110001c3516e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.207970] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.219125] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.229224] 4 locks held by python3/5517:
[ 1113.234518] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.244406] #1: ff11000180a9c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.255363] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.266517] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.276604]
[ 1113.279069] =============================================\x0a
[ 1235.446044] INFO: task python3:5517 blocked for more than 368 seconds.
[ 1235.454192] Not tainted 7.1.0-rc6-cysg-g81483f60bc8a #1
[ 1235.461445] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.471036] task:python3 state:D stack:25672 pid:5517 tgid:5517 ppid:5516 task_flags:0x400100 flags:0x00080002
[ 1235.484241] Call Trace:
[ 1235.487780]
[ 1235.490929] __schedule+0xcc3/0x19c0
[ 1235.495744] ? io_schedule_timeout+0x140/0x140
[ 1235.501525] ? find_held_lock+0x2b/0x80
[ 1235.506629] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.512515] schedule+0xd8/0x260
[ 1235.516934] schedule_preempt_disabled+0x18/0x30
[ 1235.522910] __mutex_lock+0x6af/0x2010
[ 1235.527914] ? threaded_store+0x132/0x220
[ 1235.533213] ? ww_mutex_lock+0x160/0x160
[ 1235.538409] ? selinux_inode_free_security+0x330/0x330
[ 1235.544981] ? cap_capable+0x1d6/0x3e0
[ 1235.549986] ? threaded_store+0x132/0x220
[ 1235.555271] threaded_store+0x132/0x220
[ 1235.560373] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.565667] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.570968] ? sysfs_kf_write+0x65/0x160
[ 1235.576171] ? sysfs_file_ops+0x140/0x140
[ 1235.581466] kernfs_fop_write_iter+0x335/0x510
[ 1235.587251] ? kernfs_ops+0xf0/0xf0
[ 1235.591961] new_sync_write+0x333/0x750
[ 1235.597063] ? __lock_acquire+0x508/0xc10
[ 1235.602356] ? new_sync_read+0x740/0x740
[ 1235.607556] ? lock_acquire.part.0+0xbc/0x260
[ 1235.613243] ? ksys_write+0x116/0x250
[ 1235.618143] vfs_write+0x657/0x9b0
[ 1235.622757] ksys_write+0x116/0x250
[ 1235.627467] ? __ia32_sys_read+0xc0/0xc0
[ 1235.632667] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.638256] ? rcu_is_watching+0x15/0xd0
[ 1235.643448] do_syscall_64+0xf3/0x650
[ 1235.648355] ? trace_hardirqs_off+0xd/0x30
[ 1235.653747] ? exc_page_fault+0xda/0xf0
[ 1235.658846] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.665314] RIP: 0033:0x7fb75acb364e
[ 1235.670120] RSP: 002b:00007ffeddbc19b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.679419] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb75acb364e
[ 1235.688227] RDX: 0000000000000001 RSI: 000055d5e234abf0 RDI: 0000000000000007
[ 1235.697025] RBP: 00007ffeddbc19c0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.705830] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb75ab2c390
[ 1235.714638] R13: 000055d5e234abf0 R14: 0000000000000007 R15: 00007fb75aac1120
[ 1235.723450]
[ 1235.726748] INFO: task python3:5517 is blocked on a mutex likely owned by task python3:3478.
[ 1235.737018] task:python3 state:D stack:25672 pid:3478 tgid:3478 ppid:3477 task_flags:0x400100 flags:0x00080802
[ 1235.750218] Call Trace:
[ 1235.753758]
[ 1235.756905] __schedule+0xcc3/0x19c0
[ 1235.761716] ? io_schedule_timeout+0x140/0x140
[ 1235.767497] ? find_held_lock+0x2b/0x80
[ 1235.772599] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.778770] schedule+0xd8/0x260
[ 1235.783186] schedule_timeout+0x13f/0x290
[ 1235.788482] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.794851] ? trace_contention_end+0xb3/0x1a0
[ 1235.800636] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.806328] ? ww_mutex_lock+0x160/0x160
[ 1235.811526] msleep+0x1b/0x30
[ 1235.815651] napi_set_threaded+0x3a5/0x440
[ 1235.821043] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.826535] netif_set_threaded+0x257/0x400
[ 1235.832027] threaded_store+0x1b8/0x220
[ 1235.837130] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.842420] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.847714] ? sysfs_kf_write+0x65/0x160
[ 1235.852912] ? sysfs_file_ops+0x140/0x140
[ 1235.858206] kernfs_fop_write_iter+0x335/0x510
[ 1235.863993] ? kernfs_ops+0xf0/0xf0
[ 1235.868701] new_sync_write+0x333/0x750
[ 1235.873802] ? __lock_acquire+0x508/0xc10
[ 1235.879096] ? new_sync_read+0x740/0x740
[ 1235.884296] ? lock_acquire.part.0+0xbc/0x260
[ 1235.889973] ? ksys_write+0x116/0x250
[ 1235.894882] vfs_write+0x657/0x9b0
[ 1235.899496] ksys_write+0x116/0x250
[ 1235.904208] ? __ia32_sys_read+0xc0/0xc0
[ 1235.909405] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.914992] ? rcu_is_watching+0x15/0xd0
[ 1235.920190] do_syscall_64+0xf3/0x650
[ 1235.925093] ? irq_exit_rcu+0x1a/0x30
[ 1235.929999] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.936462] RIP: 0033:0x7f76dc6a664e
[ 1235.941270] RSP: 002b:00007ffd80fde200 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.950567] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f76dc6a664e
[ 1235.959372] RDX: 0000000000000001 RSI: 000055b8a1c3abf0 RDI: 0000000000000007
[ 1235.968179] RBP: 00007ffd80fde210 R08: 0000000000000000 R09: 0000000000000000
[ 1235.976976] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f76dc51f390
[ 1235.985784] R13: 000055b8a1c3abf0 R14: 0000000000000007 R15: 00007f76dc4b5120
[ 1235.994585]
[ 1235.997847] \x0aShowing all locks held in the system:
[ 1236.006389] 1 lock held by khungtaskd/185:
[ 1236.011771] #0: ffffffffb2cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.023242] 3 locks held by systemd-journal/805:
[ 1236.029220] #0: ff1100180e8b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.039108] #1: ffffffffb2f02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: link_path_walk+0x9a7/0x1830
[ 1236.051721] #2: ff110001a57fe020 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1236.063277] 4 locks held by python3/3478:
[ 1236.068563] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.078452] #1: ff110001c3516e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.089404] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.100562] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.110659] 4 locks held by python3/5517:
[ 1236.115957] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.125844] #1: ff11000180a9c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.136804] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.147959] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.158049]
[ 1236.160512] =============================================\x0a
[ 1358.327192] INFO: task python3:5517 blocked for more than 491 seconds.
[ 1358.335344] Not tainted 7.1.0-rc6-cysg-g81483f60bc8a #1
[ 1358.342596] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.352184] task:python3 state:D stack:25672 pid:5517 tgid:5517 ppid:5516 task_flags:0x400100 flags:0x00080002
[ 1358.365389] Call Trace:
[ 1358.368932]
[ 1358.372082] __schedule+0xcc3/0x19c0
[ 1358.376899] ? io_schedule_timeout+0x140/0x140
[ 1358.382683] ? find_held_lock+0x2b/0x80
[ 1358.387789] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.393682] schedule+0xd8/0x260
[ 1358.398117] schedule_preempt_disabled+0x18/0x30
[ 1358.404099] __mutex_lock+0x6af/0x2010
[ 1358.409108] ? threaded_store+0x132/0x220
[ 1358.414408] ? ww_mutex_lock+0x160/0x160
[ 1358.419608] ? selinux_inode_free_security+0x330/0x330
[ 1358.426179] ? cap_capable+0x1d6/0x3e0
[ 1358.431187] ? threaded_store+0x132/0x220
[ 1358.436485] threaded_store+0x132/0x220
[ 1358.441587] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.446885] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.452187] ? sysfs_kf_write+0x65/0x160
[ 1358.457388] ? sysfs_file_ops+0x140/0x140
[ 1358.462685] kernfs_fop_write_iter+0x335/0x510
[ 1358.468464] ? kernfs_ops+0xf0/0xf0
[ 1358.473176] new_sync_write+0x333/0x750
[ 1358.478282] ? __lock_acquire+0x508/0xc10
[ 1358.483578] ? new_sync_read+0x740/0x740
[ 1358.488780] ? lock_acquire.part.0+0xbc/0x260
[ 1358.494466] ? ksys_write+0x116/0x250
[ 1358.499377] vfs_write+0x657/0x9b0
[ 1358.503996] ksys_write+0x116/0x250
[ 1358.508710] ? __ia32_sys_read+0xc0/0xc0
[ 1358.513908] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.519498] ? rcu_is_watching+0x15/0xd0
[ 1358.524698] do_syscall_64+0xf3/0x650
[ 1358.529603] ? trace_hardirqs_off+0xd/0x30
[ 1358.534999] ? exc_page_fault+0xda/0xf0
[ 1358.540101] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.546568] RIP: 0033:0x7fb75acb364e
[ 1358.551369] RSP: 002b:00007ffeddbc19b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.560669] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb75acb364e
[ 1358.569480] RDX: 0000000000000001 RSI: 000055d5e234abf0 RDI: 0000000000000007
[ 1358.578290] RBP: 00007ffeddbc19c0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.587097] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb75ab2c390
[ 1358.595907] R13: 000055d5e234abf0 R14: 0000000000000007 R15: 00007fb75aac1120
[ 1358.604722]
[ 1358.608022] INFO: task python3:5517 is blocked on a mutex likely owned by task python3:3478.
[ 1358.618288] task:python3 state:D stack:25672 pid:3478 tgid:3478 ppid:3477 task_flags:0x400100 flags:0x00080802
[ 1358.631494] Call Trace:
[ 1358.635037]
[ 1358.638187] __schedule+0xcc3/0x19c0
[ 1358.642999] ? io_schedule_timeout+0x140/0x140
[ 1358.648783] ? find_held_lock+0x2b/0x80
[ 1358.653878] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.660059] schedule+0xd8/0x260
[ 1358.664478] schedule_timeout+0x13f/0x290
[ 1358.669777] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.676147] ? trace_contention_end+0xb3/0x1a0
[ 1358.681931] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.687623] ? ww_mutex_lock+0x160/0x160
[ 1358.692823] msleep+0x1b/0x30
[ 1358.696941] napi_set_threaded+0x3a5/0x440
[ 1358.702331] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.707825] netif_set_threaded+0x257/0x400
[ 1358.713325] threaded_store+0x1b8/0x220
[ 1358.718425] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.723726] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.729025] ? sysfs_kf_write+0x65/0x160
[ 1358.734226] ? sysfs_file_ops+0x140/0x140
[ 1358.739520] kernfs_fop_write_iter+0x335/0x510
[ 1358.745308] ? kernfs_ops+0xf0/0xf0
[ 1358.750019] new_sync_write+0x333/0x750
[ 1358.755124] ? __lock_acquire+0x508/0xc10
[ 1358.760420] ? new_sync_read+0x740/0x740
[ 1358.765622] ? lock_acquire.part.0+0xbc/0x260
[ 1358.771307] ? ksys_write+0x116/0x250
[ 1358.776217] vfs_write+0x657/0x9b0
[ 1358.780835] ksys_write+0x116/0x250
[ 1358.785548] ? __ia32_sys_read+0xc0/0xc0
[ 1358.790748] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.796336] ? rcu_is_watching+0x15/0xd0
[ 1358.801535] do_syscall_64+0xf3/0x650
[ 1358.806443] ? irq_exit_rcu+0x1a/0x30
[ 1358.811349] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.817817] RIP: 0033:0x7f76dc6a664e
[ 1358.822616] RSP: 002b:00007ffd80fde200 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.831915] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f76dc6a664e
[ 1358.840725] RDX: 0000000000000001 RSI: 000055b8a1c3abf0 RDI: 0000000000000007
[ 1358.849534] RBP: 00007ffd80fde210 R08: 0000000000000000 R09: 0000000000000000
[ 1358.858341] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f76dc51f390
[ 1358.867141] R13: 000055b8a1c3abf0 R14: 0000000000000007 R15: 00007f76dc4b5120
[ 1358.875953]
[ 1358.879222] \x0aShowing all locks held in the system:
[ 1358.887764] 1 lock held by khungtaskd/185:
[ 1358.893160] #0: ffffffffb2cfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.904635] 2 locks held by systemd-journal/805:
[ 1358.910613] #0: ff1100180e3b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.920500] #1: ffffffffb2f02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.933318] 4 locks held by python3/3478:
[ 1358.938612] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.948502] #1: ff110001c3516e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.959453] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.970613] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.980715] 4 locks held by python3/5517:
[ 1358.986011] #0: ff110001305ac3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.995900] #1: ff11000180a9c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.006863] #2: ff11000123b38b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.018021] #3: ff11000123b71d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.028113]
[ 1359.030580] =============================================\x0a