[ 766.434707] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 766.437324] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 766.570826] kselftest: Running tests in drivers/net
[ 989.736217] INFO: task python3:5455 blocked for more than 122 seconds.
[ 989.744436] Not tainted 7.1.0-rc7-flzg-gaf33feb37e1f #1
[ 989.752310] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.761900] task:python3 state:D stack:25672 pid:5455 tgid:5455 ppid:5454 task_flags:0x400100 flags:0x00080000
[ 989.775106] Call Trace:
[ 989.778643]
[ 989.781794] __schedule+0xcc3/0x19c0
[ 989.786612] ? io_schedule_timeout+0x140/0x140
[ 989.792394] ? find_held_lock+0x2b/0x80
[ 989.797500] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.803385] schedule+0xd8/0x260
[ 989.807807] schedule_preempt_disabled+0x18/0x30
[ 989.813786] __mutex_lock+0x6af/0x2010
[ 989.818785] ? threaded_store+0x132/0x220
[ 989.824083] ? ww_mutex_lock+0x160/0x160
[ 989.829277] ? selinux_inode_free_security+0x330/0x330
[ 989.835849] ? cap_capable+0x1d6/0x3e0
[ 989.840942] ? threaded_store+0x132/0x220
[ 989.846237] threaded_store+0x132/0x220
[ 989.851342] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.856640] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.861940] ? sysfs_kf_write+0x65/0x160
[ 989.867142] ? sysfs_file_ops+0x140/0x140
[ 989.872437] kernfs_fop_write_iter+0x335/0x510
[ 989.878220] ? kernfs_ops+0xf0/0xf0
[ 989.882934] new_sync_write+0x333/0x750
[ 989.888039] ? __lock_acquire+0x508/0xc10
[ 989.893327] ? new_sync_read+0x740/0x740
[ 989.898522] ? lock_acquire.part.0+0xbc/0x260
[ 989.904207] ? ksys_write+0x116/0x250
[ 989.909117] vfs_write+0x657/0x9b0
[ 989.913732] ksys_write+0x116/0x250
[ 989.918433] ? __ia32_sys_read+0xc0/0xc0
[ 989.923633] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.929220] ? rcu_is_watching+0x15/0xd0
[ 989.934421] do_syscall_64+0xf3/0x650
[ 989.939328] ? trace_hardirqs_off+0xd/0x30
[ 989.944723] ? exc_page_fault+0xda/0xf0
[ 989.949826] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.956294] RIP: 0033:0x7fa1470a964e
[ 989.961103] RSP: 002b:00007ffdb8378d70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.970403] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1470a964e
[ 989.979213] RDX: 0000000000000001 RSI: 00005572b0e69bf0 RDI: 0000000000000007
[ 989.988023] RBP: 00007ffdb8378d80 R08: 0000000000000000 R09: 0000000000000000
[ 989.996832] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa146f22390
[ 990.005640] R13: 00005572b0e69bf0 R14: 0000000000000007 R15: 00007fa146eb5120
[ 990.014463]
[ 990.017749] INFO: task python3:5455 is blocked on a mutex likely owned by task python3:3461.
[ 990.028023] task:python3 state:D stack:25672 pid:3461 tgid:3461 ppid:3460 task_flags:0x400100 flags:0x00080802
[ 990.041227] Call Trace:
[ 990.044767]
[ 990.047917] __schedule+0xcc3/0x19c0
[ 990.052731] ? io_schedule_timeout+0x140/0x140
[ 990.058518] ? find_held_lock+0x2b/0x80
[ 990.063618] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.069797] schedule+0xd8/0x260
[ 990.074216] schedule_timeout+0x13f/0x290
[ 990.079513] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.085875] ? trace_contention_end+0xb3/0x1a0
[ 990.091659] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.097352] ? ww_mutex_lock+0x160/0x160
[ 990.102554] msleep+0x1b/0x30
[ 990.106681] napi_set_threaded+0x3a5/0x440
[ 990.112078] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.117573] netif_set_threaded+0x257/0x400
[ 990.123070] threaded_store+0x1b8/0x220
[ 990.128170] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.133469] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.138765] ? sysfs_kf_write+0x65/0x160
[ 990.143964] ? sysfs_file_ops+0x140/0x140
[ 990.149252] kernfs_fop_write_iter+0x335/0x510
[ 990.155036] ? kernfs_ops+0xf0/0xf0
[ 990.159749] new_sync_write+0x333/0x750
[ 990.164843] ? __lock_acquire+0x508/0xc10
[ 990.170139] ? new_sync_read+0x740/0x740
[ 990.175338] ? lock_acquire.part.0+0xbc/0x260
[ 990.181025] ? ksys_write+0x116/0x250
[ 990.185934] vfs_write+0x657/0x9b0
[ 990.190548] ksys_write+0x116/0x250
[ 990.195262] ? __ia32_sys_read+0xc0/0xc0
[ 990.200461] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.206058] ? rcu_is_watching+0x15/0xd0
[ 990.211260] do_syscall_64+0xf3/0x650
[ 990.216165] ? trace_hardirqs_off+0xd/0x30
[ 990.221561] ? exc_page_fault+0xda/0xf0
[ 990.226664] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.233130] RIP: 0033:0x7fc142b9064e
[ 990.237938] RSP: 002b:00007ffe0ce67bb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.247234] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc142b9064e
[ 990.256036] RDX: 0000000000000001 RSI: 000055ca2d862bf0 RDI: 0000000000000007
[ 990.264845] RBP: 00007ffe0ce67bc0 R08: 0000000000000000 R09: 0000000000000000
[ 990.273652] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc142a09390
[ 990.282461] R13: 000055ca2d862bf0 R14: 0000000000000007 R15: 00007fc14299d120
[ 990.291273]
[ 990.294533] \x0aShowing all locks held in the system:
[ 990.303075] 1 lock held by khungtaskd/185:
[ 990.308467] #0: ffffffff988fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.319939] 3 locks held by systemd-journal/801:
[ 990.325917] #0: ff1100180dab74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.335805] #1: ffffffff98b02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.348613] #2: ff110001a338fd00 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.360172] 4 locks held by python3/3461:
[ 990.365468] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.375356] #1: ff11000144c61480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.386320] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.397478] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.407566] 4 locks held by python3/5455:
[ 990.412861] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.422750] #1: ff110001ab626280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.433712] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.444871] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.454958] 5 locks held by systemd/6174:
[ 990.460255] #0: ff11000130ba93e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.470145] #1: ff11000185ffa080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.481104] #2: ffffffff98969c20 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 990.491774] #3: ffffffff987687b0 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 990.503416] #4: ffffffff98969990 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 990.515846]
[ 990.518312] =============================================\x0a
[ 1112.618275] INFO: task python3:5455 blocked for more than 245 seconds.
[ 1112.626424] Not tainted 7.1.0-rc7-flzg-gaf33feb37e1f #1
[ 1112.633672] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.643254] task:python3 state:D stack:25672 pid:5455 tgid:5455 ppid:5454 task_flags:0x400100 flags:0x00080000
[ 1112.656462] Call Trace:
[ 1112.660002]
[ 1112.663152] __schedule+0xcc3/0x19c0
[ 1112.667970] ? io_schedule_timeout+0x140/0x140
[ 1112.673757] ? find_held_lock+0x2b/0x80
[ 1112.678864] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.684749] schedule+0xd8/0x260
[ 1112.689167] schedule_preempt_disabled+0x18/0x30
[ 1112.695146] __mutex_lock+0x6af/0x2010
[ 1112.700152] ? threaded_store+0x132/0x220
[ 1112.705450] ? ww_mutex_lock+0x160/0x160
[ 1112.710650] ? selinux_inode_free_security+0x330/0x330
[ 1112.717219] ? cap_capable+0x1d6/0x3e0
[ 1112.722228] ? threaded_store+0x132/0x220
[ 1112.727528] threaded_store+0x132/0x220
[ 1112.732631] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.737929] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.743229] ? sysfs_kf_write+0x65/0x160
[ 1112.748420] ? sysfs_file_ops+0x140/0x140
[ 1112.753717] kernfs_fop_write_iter+0x335/0x510
[ 1112.759495] ? kernfs_ops+0xf0/0xf0
[ 1112.764208] new_sync_write+0x333/0x750
[ 1112.769313] ? __lock_acquire+0x508/0xc10
[ 1112.774601] ? new_sync_read+0x740/0x740
[ 1112.779803] ? lock_acquire.part.0+0xbc/0x260
[ 1112.785491] ? ksys_write+0x116/0x250
[ 1112.790393] vfs_write+0x657/0x9b0
[ 1112.795010] ksys_write+0x116/0x250
[ 1112.799720] ? __ia32_sys_read+0xc0/0xc0
[ 1112.804919] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.810501] ? rcu_is_watching+0x15/0xd0
[ 1112.815705] do_syscall_64+0xf3/0x650
[ 1112.820611] ? trace_hardirqs_off+0xd/0x30
[ 1112.825999] ? exc_page_fault+0xda/0xf0
[ 1112.831102] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.837573] RIP: 0033:0x7fa1470a964e
[ 1112.842383] RSP: 002b:00007ffdb8378d70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.851680] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1470a964e
[ 1112.860481] RDX: 0000000000000001 RSI: 00005572b0e69bf0 RDI: 0000000000000007
[ 1112.869289] RBP: 00007ffdb8378d80 R08: 0000000000000000 R09: 0000000000000000
[ 1112.878089] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa146f22390
[ 1112.886899] R13: 00005572b0e69bf0 R14: 0000000000000007 R15: 00007fa146eb5120
[ 1112.895712]
[ 1112.898983] INFO: task python3:5455 is blocked on a mutex likely owned by task python3:3461.
[ 1112.909254] task:python3 state:D stack:25672 pid:3461 tgid:3461 ppid:3460 task_flags:0x400100 flags:0x00080802
[ 1112.922458] Call Trace:
[ 1112.925997]
[ 1112.929147] __schedule+0xcc3/0x19c0
[ 1112.933958] ? io_schedule_timeout+0x140/0x140
[ 1112.939745] ? find_held_lock+0x2b/0x80
[ 1112.944845] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.951025] schedule+0xd8/0x260
[ 1112.955444] schedule_timeout+0x13f/0x290
[ 1112.960744] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.967118] ? trace_contention_end+0xb3/0x1a0
[ 1112.972904] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.978595] ? ww_mutex_lock+0x160/0x160
[ 1112.983796] msleep+0x1b/0x30
[ 1112.987924] napi_set_threaded+0x3a5/0x440
[ 1112.993318] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.998812] netif_set_threaded+0x257/0x400
[ 1113.004306] threaded_store+0x1b8/0x220
[ 1113.009407] ? xps_rxqs_store+0x2e0/0x2e0
[ 1113.014706] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.020001] ? sysfs_kf_write+0x65/0x160
[ 1113.025204] ? sysfs_file_ops+0x140/0x140
[ 1113.030499] kernfs_fop_write_iter+0x335/0x510
[ 1113.036282] ? kernfs_ops+0xf0/0xf0
[ 1113.040995] new_sync_write+0x333/0x750
[ 1113.046101] ? __lock_acquire+0x508/0xc10
[ 1113.051398] ? new_sync_read+0x740/0x740
[ 1113.056588] ? lock_acquire.part.0+0xbc/0x260
[ 1113.062275] ? ksys_write+0x116/0x250
[ 1113.067185] vfs_write+0x657/0x9b0
[ 1113.071800] ksys_write+0x116/0x250
[ 1113.076503] ? __ia32_sys_read+0xc0/0xc0
[ 1113.081703] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.087284] ? rcu_is_watching+0x15/0xd0
[ 1113.092476] do_syscall_64+0xf3/0x650
[ 1113.097383] ? trace_hardirqs_off+0xd/0x30
[ 1113.102770] ? exc_page_fault+0xda/0xf0
[ 1113.107871] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.114337] RIP: 0033:0x7fc142b9064e
[ 1113.119148] RSP: 002b:00007ffe0ce67bb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.128445] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc142b9064e
[ 1113.137251] RDX: 0000000000000001 RSI: 000055ca2d862bf0 RDI: 0000000000000007
[ 1113.146061] RBP: 00007ffe0ce67bc0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.154868] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc142a09390
[ 1113.163676] R13: 000055ca2d862bf0 R14: 0000000000000007 R15: 00007fc14299d120
[ 1113.172480]
[ 1113.175742] \x0aShowing all locks held in the system:
[ 1113.184279] 1 lock held by khungtaskd/185:
[ 1113.189672] #0: ffffffff988fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.201155] 4 locks held by python3/3461:
[ 1113.206444] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.216333] #1: ff11000144c61480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.227297] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.238453] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.248544] 4 locks held by python3/5455:
[ 1113.253840] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.263729] #1: ff110001ab626280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.274691] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.285849] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.295938]
[ 1113.298402] =============================================\x0a
[ 1235.498347] INFO: task python3:5455 blocked for more than 368 seconds.
[ 1235.506499] Not tainted 7.1.0-rc7-flzg-gaf33feb37e1f #1
[ 1235.513750] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.523338] task:python3 state:D stack:25672 pid:5455 tgid:5455 ppid:5454 task_flags:0x400100 flags:0x00080002
[ 1235.536545] Call Trace:
[ 1235.540083]
[ 1235.543230] __schedule+0xcc3/0x19c0
[ 1235.548048] ? io_schedule_timeout+0x140/0x140
[ 1235.553833] ? find_held_lock+0x2b/0x80
[ 1235.558937] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.564823] schedule+0xd8/0x260
[ 1235.569243] schedule_preempt_disabled+0x18/0x30
[ 1235.575214] __mutex_lock+0x6af/0x2010
[ 1235.580211] ? threaded_store+0x132/0x220
[ 1235.585511] ? ww_mutex_lock+0x160/0x160
[ 1235.590708] ? selinux_inode_free_security+0x330/0x330
[ 1235.597280] ? cap_capable+0x1d6/0x3e0
[ 1235.602285] ? threaded_store+0x132/0x220
[ 1235.607582] threaded_store+0x132/0x220
[ 1235.612678] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.617976] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.623274] ? sysfs_kf_write+0x65/0x160
[ 1235.628474] ? sysfs_file_ops+0x140/0x140
[ 1235.633771] kernfs_fop_write_iter+0x335/0x510
[ 1235.639549] ? kernfs_ops+0xf0/0xf0
[ 1235.644261] new_sync_write+0x333/0x750
[ 1235.649365] ? __lock_acquire+0x508/0xc10
[ 1235.654654] ? new_sync_read+0x740/0x740
[ 1235.659854] ? lock_acquire.part.0+0xbc/0x260
[ 1235.665540] ? ksys_write+0x116/0x250
[ 1235.670450] vfs_write+0x657/0x9b0
[ 1235.675064] ksys_write+0x116/0x250
[ 1235.679777] ? __ia32_sys_read+0xc0/0xc0
[ 1235.684964] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.690554] ? rcu_is_watching+0x15/0xd0
[ 1235.695754] do_syscall_64+0xf3/0x650
[ 1235.700660] ? trace_hardirqs_off+0xd/0x30
[ 1235.706055] ? exc_page_fault+0xda/0xf0
[ 1235.711155] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.717623] RIP: 0033:0x7fa1470a964e
[ 1235.722430] RSP: 002b:00007ffdb8378d70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.731728] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1470a964e
[ 1235.740535] RDX: 0000000000000001 RSI: 00005572b0e69bf0 RDI: 0000000000000007
[ 1235.749342] RBP: 00007ffdb8378d80 R08: 0000000000000000 R09: 0000000000000000
[ 1235.758151] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa146f22390
[ 1235.766959] R13: 00005572b0e69bf0 R14: 0000000000000007 R15: 00007fa146eb5120
[ 1235.775773]
[ 1235.779035] INFO: task python3:5455 is blocked on a mutex likely owned by task python3:3461.
[ 1235.789308] task:python3 state:D stack:25672 pid:3461 tgid:3461 ppid:3460 task_flags:0x400100 flags:0x00080802
[ 1235.802513] Call Trace:
[ 1235.806049]
[ 1235.809199] __schedule+0xcc3/0x19c0
[ 1235.814008] ? io_schedule_timeout+0x140/0x140
[ 1235.819791] ? find_held_lock+0x2b/0x80
[ 1235.824893] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.831069] schedule+0xd8/0x260
[ 1235.835479] schedule_timeout+0x13f/0x290
[ 1235.840775] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.847143] ? trace_contention_end+0xb3/0x1a0
[ 1235.852929] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.858609] ? ww_mutex_lock+0x160/0x160
[ 1235.863808] msleep+0x1b/0x30
[ 1235.867933] napi_set_threaded+0x3a5/0x440
[ 1235.873326] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.878821] netif_set_threaded+0x257/0x400
[ 1235.884313] threaded_store+0x1b8/0x220
[ 1235.889415] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.894704] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.899993] ? sysfs_kf_write+0x65/0x160
[ 1235.905195] ? sysfs_file_ops+0x140/0x140
[ 1235.910490] kernfs_fop_write_iter+0x335/0x510
[ 1235.916274] ? kernfs_ops+0xf0/0xf0
[ 1235.920984] new_sync_write+0x333/0x750
[ 1235.926085] ? __lock_acquire+0x508/0xc10
[ 1235.931382] ? new_sync_read+0x740/0x740
[ 1235.936653] ? lock_acquire.part.0+0xbc/0x260
[ 1235.942338] ? ksys_write+0x116/0x250
[ 1235.947248] vfs_write+0x657/0x9b0
[ 1235.951863] ksys_write+0x116/0x250
[ 1235.956577] ? __ia32_sys_read+0xc0/0xc0
[ 1235.961775] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.967363] ? rcu_is_watching+0x15/0xd0
[ 1235.972562] do_syscall_64+0xf3/0x650
[ 1235.977467] ? trace_hardirqs_off+0xd/0x30
[ 1235.982862] ? exc_page_fault+0xda/0xf0
[ 1235.987963] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.994431] RIP: 0033:0x7fc142b9064e
[ 1235.999239] RSP: 002b:00007ffe0ce67bb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.008537] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc142b9064e
[ 1236.017346] RDX: 0000000000000001 RSI: 000055ca2d862bf0 RDI: 0000000000000007
[ 1236.026227] RBP: 00007ffe0ce67bc0 R08: 0000000000000000 R09: 0000000000000000
[ 1236.035036] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc142a09390
[ 1236.043842] R13: 000055ca2d862bf0 R14: 0000000000000007 R15: 00007fc14299d120
[ 1236.052654]
[ 1236.055918] \x0aShowing all locks held in the system:
[ 1236.064455] 1 lock held by khungtaskd/185:
[ 1236.069908] #0: ffffffff988fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.081382] 3 locks held by systemd-journal/801:
[ 1236.087359] #0: ff1100180de374e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.097248] #1: ffffffff98b02ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.110062] #2: ff110001a338fd00 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1236.121619] 4 locks held by python3/3461:
[ 1236.126912] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.136797] #1: ff11000144c61480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.147757] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.158974] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.169065] 4 locks held by python3/5455:
[ 1236.174352] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.184238] #1: ff110001ab626280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.195266] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.206420] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.216508]
[ 1236.218972] =============================================\x0a
[ 1358.380456] INFO: task python3:5455 blocked for more than 491 seconds.
[ 1358.388604] Not tainted 7.1.0-rc7-flzg-gaf33feb37e1f #1
[ 1358.395855] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.405446] task:python3 state:D stack:25672 pid:5455 tgid:5455 ppid:5454 task_flags:0x400100 flags:0x00080002
[ 1358.418654] Call Trace:
[ 1358.422195]
[ 1358.425349] __schedule+0xcc3/0x19c0
[ 1358.430167] ? io_schedule_timeout+0x140/0x140
[ 1358.435955] ? find_held_lock+0x2b/0x80
[ 1358.441060] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.446945] schedule+0xd8/0x260
[ 1358.451363] schedule_preempt_disabled+0x18/0x30
[ 1358.457343] __mutex_lock+0x6af/0x2010
[ 1358.462347] ? threaded_store+0x132/0x220
[ 1358.467645] ? ww_mutex_lock+0x160/0x160
[ 1358.472836] ? selinux_inode_free_security+0x330/0x330
[ 1358.479405] ? cap_capable+0x1d6/0x3e0
[ 1358.484414] ? threaded_store+0x132/0x220
[ 1358.489711] threaded_store+0x132/0x220
[ 1358.494812] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.500110] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.505410] ? sysfs_kf_write+0x65/0x160
[ 1358.510613] ? sysfs_file_ops+0x140/0x140
[ 1358.515909] kernfs_fop_write_iter+0x335/0x510
[ 1358.521693] ? kernfs_ops+0xf0/0xf0
[ 1358.526404] new_sync_write+0x333/0x750
[ 1358.531509] ? __lock_acquire+0x508/0xc10
[ 1358.536805] ? new_sync_read+0x740/0x740
[ 1358.542008] ? lock_acquire.part.0+0xbc/0x260
[ 1358.547695] ? ksys_write+0x116/0x250
[ 1358.552603] vfs_write+0x657/0x9b0
[ 1358.557219] ksys_write+0x116/0x250
[ 1358.561933] ? __ia32_sys_read+0xc0/0xc0
[ 1358.567133] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.572723] ? rcu_is_watching+0x15/0xd0
[ 1358.577925] do_syscall_64+0xf3/0x650
[ 1358.582833] ? trace_hardirqs_off+0xd/0x30
[ 1358.588227] ? exc_page_fault+0xda/0xf0
[ 1358.593330] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.599799] RIP: 0033:0x7fa1470a964e
[ 1358.604610] RSP: 002b:00007ffdb8378d70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.613907] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1470a964e
[ 1358.622708] RDX: 0000000000000001 RSI: 00005572b0e69bf0 RDI: 0000000000000007
[ 1358.631518] RBP: 00007ffdb8378d80 R08: 0000000000000000 R09: 0000000000000000
[ 1358.640326] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa146f22390
[ 1358.649126] R13: 00005572b0e69bf0 R14: 0000000000000007 R15: 00007fa146eb5120
[ 1358.657939]
[ 1358.661201] INFO: task python3:5455 is blocked on a mutex likely owned by task python3:3461.
[ 1358.671475] task:python3 state:D stack:25672 pid:3461 tgid:3461 ppid:3460 task_flags:0x400100 flags:0x00080802
[ 1358.684678] Call Trace:
[ 1358.688217]
[ 1358.691370] __schedule+0xcc3/0x19c0
[ 1358.696182] ? io_schedule_timeout+0x140/0x140
[ 1358.701968] ? find_held_lock+0x2b/0x80
[ 1358.707071] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.713249] schedule+0xd8/0x260
[ 1358.717668] schedule_timeout+0x13f/0x290
[ 1358.722965] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.729335] ? trace_contention_end+0xb3/0x1a0
[ 1358.735122] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.740813] ? ww_mutex_lock+0x160/0x160
[ 1358.746014] msleep+0x1b/0x30
[ 1358.750139] napi_set_threaded+0x3a5/0x440
[ 1358.755533] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.761030] netif_set_threaded+0x257/0x400
[ 1358.766524] threaded_store+0x1b8/0x220
[ 1358.771625] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.776922] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.782211] ? sysfs_kf_write+0x65/0x160
[ 1358.787411] ? sysfs_file_ops+0x140/0x140
[ 1358.792707] kernfs_fop_write_iter+0x335/0x510
[ 1358.798491] ? kernfs_ops+0xf0/0xf0
[ 1358.803202] new_sync_write+0x333/0x750
[ 1358.808309] ? __lock_acquire+0x508/0xc10
[ 1358.813605] ? new_sync_read+0x740/0x740
[ 1358.818806] ? lock_acquire.part.0+0xbc/0x260
[ 1358.824492] ? ksys_write+0x116/0x250
[ 1358.829402] vfs_write+0x657/0x9b0
[ 1358.834017] ksys_write+0x116/0x250
[ 1358.838730] ? __ia32_sys_read+0xc0/0xc0
[ 1358.843918] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.849509] ? rcu_is_watching+0x15/0xd0
[ 1358.854711] do_syscall_64+0xf3/0x650
[ 1358.859617] ? trace_hardirqs_off+0xd/0x30
[ 1358.865011] ? exc_page_fault+0xda/0xf0
[ 1358.870114] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.876582] RIP: 0033:0x7fc142b9064e
[ 1358.881382] RSP: 002b:00007ffe0ce67bb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.890678] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc142b9064e
[ 1358.899479] RDX: 0000000000000001 RSI: 000055ca2d862bf0 RDI: 0000000000000007
[ 1358.908288] RBP: 00007ffe0ce67bc0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.917095] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc142a09390
[ 1358.925904] R13: 000055ca2d862bf0 R14: 0000000000000007 R15: 00007fc14299d120
[ 1358.934717]
[ 1358.937985] \x0aShowing all locks held in the system:
[ 1358.946521] 1 lock held by khungtaskd/185:
[ 1358.951913] #0: ffffffff988fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.963393] 4 locks held by python3/3461:
[ 1358.968690] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.978580] #1: ff11000144c61480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.989543] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.000699] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.010788] 4 locks held by python3/5455:
[ 1359.016086] #0: ff110001309593e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.025974] #1: ff110001ab626280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.036935] #2: ff11000123464b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.048092] #3: ff110001234d1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.058186]
[ 1359.060653] =============================================\x0a