[ 780.704275] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 780.706606] nipa-hw-worker: [5/45] Retrying drivers/net:napi_threaded.py
[ 780.857838] kselftest: Running tests in drivers/net
[ 989.707859] INFO: task python3:5717 blocked for more than 122 seconds.
[ 989.716086] Not tainted 7.1.0-rc1-qxpi-gf298010a814b #1
[ 989.723908] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.733500] task:python3 state:D stack:25672 pid:5717 tgid:5717 ppid:5716 task_flags:0x400100 flags:0x00080000
[ 989.746709] Call Trace:
[ 989.750249]
[ 989.753402] __schedule+0xcba/0x1940
[ 989.758219] ? io_schedule_timeout+0x140/0x140
[ 989.764003] ? find_held_lock+0x2b/0x80
[ 989.769099] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.774976] schedule+0xd4/0x260
[ 989.779395] schedule_preempt_disabled+0x18/0x30
[ 989.785375] __mutex_lock+0x6a3/0x2000
[ 989.790372] ? threaded_store+0x132/0x220
[ 989.795674] ? ww_mutex_lock+0x160/0x160
[ 989.800877] ? selinux_inode_free_security+0x330/0x330
[ 989.807447] ? cap_capable+0x1d6/0x3e0
[ 989.812458] ? threaded_store+0x132/0x220
[ 989.817755] threaded_store+0x132/0x220
[ 989.822861] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.828157] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.833446] ? sysfs_kf_write+0x65/0x160
[ 989.838646] ? sysfs_file_ops+0x140/0x140
[ 989.843944] kernfs_fop_write_iter+0x335/0x510
[ 989.849727] ? kernfs_ops+0xf0/0xf0
[ 989.854440] new_sync_write+0x333/0x750
[ 989.859546] ? __lock_acquire+0x508/0xc10
[ 989.864846] ? new_sync_read+0x740/0x740
[ 989.870047] ? lock_acquire.part.0+0xbc/0x260
[ 989.875734] ? ksys_write+0x116/0x250
[ 989.880647] vfs_write+0x657/0x9b0
[ 989.885264] ksys_write+0x116/0x250
[ 989.889975] ? __ia32_sys_read+0xc0/0xc0
[ 989.895175] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.900765] ? rcu_is_watching+0x15/0xd0
[ 989.905969] do_syscall_64+0xf3/0x650
[ 989.910875] ? trace_hardirqs_off+0xd/0x30
[ 989.916262] ? exc_page_fault+0xda/0xf0
[ 989.921363] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.927832] RIP: 0033:0x7f5f84ee422e
[ 989.932642] RSP: 002b:00007ffef5296d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.941942] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5f84ee422e
[ 989.950750] RDX: 0000000000000001 RSI: 00005556e9ea0e10 RDI: 0000000000000007
[ 989.959560] RBP: 00007ffef5296d40 R08: 0000000000000000 R09: 0000000000000000
[ 989.968368] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5f84d5c390
[ 989.977176] R13: 00005556e9ea0e10 R14: 0000000000000007 R15: 0000000000000003
[ 989.985990]
[ 989.989277] INFO: task python3:5717 is blocked on a mutex likely owned by task python3:3628.
[ 989.999551] task:python3 state:D stack:25672 pid:3628 tgid:3628 ppid:3627 task_flags:0x400100 flags:0x00080802
[ 990.012753] Call Trace:
[ 990.016293]
[ 990.019445] __schedule+0xcba/0x1940
[ 990.024256] ? io_schedule_timeout+0x140/0x140
[ 990.030043] ? find_held_lock+0x2b/0x80
[ 990.035147] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.041330] schedule+0xd4/0x260
[ 990.045747] schedule_timeout+0x13f/0x290
[ 990.051046] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.057415] ? trace_contention_end+0xb3/0x1a0
[ 990.063199] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.068891] ? ww_mutex_lock+0x160/0x160
[ 990.074100] msleep+0x1b/0x30
[ 990.078225] napi_set_threaded+0x3a5/0x440
[ 990.083622] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.089119] netif_set_threaded+0x257/0x400
[ 990.094612] threaded_store+0x1b8/0x220
[ 990.099713] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.105009] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.110307] ? sysfs_kf_write+0x65/0x160
[ 990.115505] ? sysfs_file_ops+0x140/0x140
[ 990.120803] kernfs_fop_write_iter+0x335/0x510
[ 990.126581] ? kernfs_ops+0xf0/0xf0
[ 990.131292] new_sync_write+0x333/0x750
[ 990.136393] ? __lock_acquire+0x508/0xc10
[ 990.141694] ? new_sync_read+0x740/0x740
[ 990.146893] ? lock_acquire.part.0+0xbc/0x260
[ 990.152582] ? ksys_write+0x116/0x250
[ 990.157493] vfs_write+0x657/0x9b0
[ 990.162111] ksys_write+0x116/0x250
[ 990.166823] ? __ia32_sys_read+0xc0/0xc0
[ 990.172023] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.177611] ? rcu_is_watching+0x15/0xd0
[ 990.182812] do_syscall_64+0xf3/0x650
[ 990.187720] ? irq_exit_rcu+0x1a/0x30
[ 990.192630] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.199096] RIP: 0033:0x7f3fe559522e
[ 990.203902] RSP: 002b:00007ffdbaf6bd50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.213201] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3fe559522e
[ 990.222011] RDX: 0000000000000001 RSI: 000055fe6c348e10 RDI: 0000000000000007
[ 990.230821] RBP: 00007ffdbaf6bd60 R08: 0000000000000000 R09: 0000000000000000
[ 990.239627] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3fe540d390
[ 990.248437] R13: 000055fe6c348e10 R14: 0000000000000007 R15: 0000000000000003
[ 990.257250]
[ 990.260508] \x0aShowing all locks held in the system:
[ 990.269044] 1 lock held by khungtaskd/185:
[ 990.274438] #0: ffffffffa0efdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.285913] 2 locks held by systemd-journal/804:
[ 990.291885] #0: ff1100180dcb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 990.301776] #1: ff110001a22c8378 (&mm->mmap_lock){++++}-{4:4}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.312556] 4 locks held by python3/3628:
[ 990.317853] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.327743] #1: ff1100015474c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.338697] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.349852] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.359948] 4 locks held by python3/5717:
[ 990.365244] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.375132] #1: ff110001dce38880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.386095] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.397252] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.407338]
[ 990.409802] =============================================\x0a
[ 1112.589085] INFO: task python3:5717 blocked for more than 245 seconds.
[ 1112.597234] Not tainted 7.1.0-rc1-qxpi-gf298010a814b #1
[ 1112.604483] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.614064] task:python3 state:D stack:25672 pid:5717 tgid:5717 ppid:5716 task_flags:0x400100 flags:0x00080000
[ 1112.627269] Call Trace:
[ 1112.630805]
[ 1112.633955] __schedule+0xcba/0x1940
[ 1112.638771] ? io_schedule_timeout+0x140/0x140
[ 1112.644557] ? find_held_lock+0x2b/0x80
[ 1112.649663] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.655547] schedule+0xd4/0x260
[ 1112.659964] schedule_preempt_disabled+0x18/0x30
[ 1112.665944] __mutex_lock+0x6a3/0x2000
[ 1112.670949] ? threaded_store+0x132/0x220
[ 1112.676246] ? ww_mutex_lock+0x160/0x160
[ 1112.681442] ? selinux_inode_free_security+0x330/0x330
[ 1112.688011] ? cap_capable+0x1d6/0x3e0
[ 1112.693026] ? threaded_store+0x132/0x220
[ 1112.698324] threaded_store+0x132/0x220
[ 1112.703420] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.708715] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.714012] ? sysfs_kf_write+0x65/0x160
[ 1112.719212] ? sysfs_file_ops+0x140/0x140
[ 1112.724500] kernfs_fop_write_iter+0x335/0x510
[ 1112.730282] ? kernfs_ops+0xf0/0xf0
[ 1112.734993] new_sync_write+0x333/0x750
[ 1112.740088] ? __lock_acquire+0x508/0xc10
[ 1112.745373] ? new_sync_read+0x740/0x740
[ 1112.750574] ? lock_acquire.part.0+0xbc/0x260
[ 1112.756261] ? ksys_write+0x116/0x250
[ 1112.761171] vfs_write+0x657/0x9b0
[ 1112.765785] ksys_write+0x116/0x250
[ 1112.770487] ? __ia32_sys_read+0xc0/0xc0
[ 1112.775683] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.781270] ? rcu_is_watching+0x15/0xd0
[ 1112.786469] do_syscall_64+0xf3/0x650
[ 1112.791372] ? trace_hardirqs_off+0xd/0x30
[ 1112.796766] ? exc_page_fault+0xda/0xf0
[ 1112.801869] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.808335] RIP: 0033:0x7f5f84ee422e
[ 1112.813143] RSP: 002b:00007ffef5296d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.822438] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5f84ee422e
[ 1112.831243] RDX: 0000000000000001 RSI: 00005556e9ea0e10 RDI: 0000000000000007
[ 1112.840049] RBP: 00007ffef5296d40 R08: 0000000000000000 R09: 0000000000000000
[ 1112.848854] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5f84d5c390
[ 1112.857664] R13: 00005556e9ea0e10 R14: 0000000000000007 R15: 0000000000000003
[ 1112.866474]
[ 1112.869756] INFO: task python3:5717 is blocked on a mutex likely owned by task python3:3628.
[ 1112.880026] task:python3 state:D stack:25672 pid:3628 tgid:3628 ppid:3627 task_flags:0x400100 flags:0x00080802
[ 1112.893229] Call Trace:
[ 1112.896768]
[ 1112.899919] __schedule+0xcba/0x1940
[ 1112.904728] ? io_schedule_timeout+0x140/0x140
[ 1112.910513] ? find_held_lock+0x2b/0x80
[ 1112.915616] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.921794] schedule+0xd4/0x260
[ 1112.926212] schedule_timeout+0x13f/0x290
[ 1112.931510] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.937870] ? trace_contention_end+0xb3/0x1a0
[ 1112.943651] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.949339] ? ww_mutex_lock+0x160/0x160
[ 1112.954537] msleep+0x1b/0x30
[ 1112.958660] napi_set_threaded+0x3a5/0x440
[ 1112.964055] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.969550] netif_set_threaded+0x257/0x400
[ 1112.975044] threaded_store+0x1b8/0x220
[ 1112.980144] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.985439] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.990733] ? sysfs_kf_write+0x65/0x160
[ 1112.995932] ? sysfs_file_ops+0x140/0x140
[ 1113.001226] kernfs_fop_write_iter+0x335/0x510
[ 1113.007010] ? kernfs_ops+0xf0/0xf0
[ 1113.011720] new_sync_write+0x333/0x750
[ 1113.016819] ? __lock_acquire+0x508/0xc10
[ 1113.022114] ? new_sync_read+0x740/0x740
[ 1113.027311] ? lock_acquire.part.0+0xbc/0x260
[ 1113.032995] ? ksys_write+0x116/0x250
[ 1113.037904] vfs_write+0x657/0x9b0
[ 1113.042518] ksys_write+0x116/0x250
[ 1113.047227] ? __ia32_sys_read+0xc0/0xc0
[ 1113.052424] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.058010] ? rcu_is_watching+0x15/0xd0
[ 1113.063209] do_syscall_64+0xf3/0x650
[ 1113.068114] ? irq_exit_rcu+0x1a/0x30
[ 1113.073020] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.079486] RIP: 0033:0x7f3fe559522e
[ 1113.084291] RSP: 002b:00007ffdbaf6bd50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.093586] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3fe559522e
[ 1113.102392] RDX: 0000000000000001 RSI: 000055fe6c348e10 RDI: 0000000000000007
[ 1113.111198] RBP: 00007ffdbaf6bd60 R08: 0000000000000000 R09: 0000000000000000
[ 1113.120006] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3fe540d390
[ 1113.128812] R13: 000055fe6c348e10 R14: 0000000000000007 R15: 0000000000000003
[ 1113.137626]
[ 1113.140887] \x0aShowing all locks held in the system:
[ 1113.149423] 1 lock held by khungtaskd/185:
[ 1113.154815] #0: ffffffffa0efdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.166285] 2 locks held by systemd-journal/804:
[ 1113.172263] #0: ff1100180e8b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1113.182147] #1: ffffffffa1103120 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.194973] 4 locks held by python3/3628:
[ 1113.200269] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.210156] #1: ff1100015474c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.221116] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.232273] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.242355] 4 locks held by python3/5717:
[ 1113.247652] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.257540] #1: ff110001dce38880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.268501] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.279657] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.289744]
[ 1113.292208] =============================================\x0a
[ 1235.470331] INFO: task python3:5717 blocked for more than 368 seconds.
[ 1235.478481] Not tainted 7.1.0-rc1-qxpi-gf298010a814b #1
[ 1235.485731] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.495322] task:python3 state:D stack:25672 pid:5717 tgid:5717 ppid:5716 task_flags:0x400100 flags:0x00080002
[ 1235.508528] Call Trace:
[ 1235.512068]
[ 1235.515220] __schedule+0xcba/0x1940
[ 1235.520038] ? io_schedule_timeout+0x140/0x140
[ 1235.525823] ? find_held_lock+0x2b/0x80
[ 1235.530931] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.536816] schedule+0xd4/0x260
[ 1235.541236] schedule_preempt_disabled+0x18/0x30
[ 1235.547218] __mutex_lock+0x6a3/0x2000
[ 1235.552223] ? threaded_store+0x132/0x220
[ 1235.557521] ? ww_mutex_lock+0x160/0x160
[ 1235.562721] ? selinux_inode_free_security+0x330/0x330
[ 1235.569290] ? cap_capable+0x1d6/0x3e0
[ 1235.574305] ? threaded_store+0x132/0x220
[ 1235.579613] threaded_store+0x132/0x220
[ 1235.584716] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.590003] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.595304] ? sysfs_kf_write+0x65/0x160
[ 1235.600507] ? sysfs_file_ops+0x140/0x140
[ 1235.605804] kernfs_fop_write_iter+0x335/0x510
[ 1235.611590] ? kernfs_ops+0xf0/0xf0
[ 1235.616300] new_sync_write+0x333/0x750
[ 1235.621396] ? __lock_acquire+0x508/0xc10
[ 1235.626686] ? new_sync_read+0x740/0x740
[ 1235.631886] ? lock_acquire.part.0+0xbc/0x260
[ 1235.637573] ? ksys_write+0x116/0x250
[ 1235.642481] vfs_write+0x657/0x9b0
[ 1235.647096] ksys_write+0x116/0x250
[ 1235.651809] ? __ia32_sys_read+0xc0/0xc0
[ 1235.657009] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.662590] ? rcu_is_watching+0x15/0xd0
[ 1235.667782] do_syscall_64+0xf3/0x650
[ 1235.672690] ? trace_hardirqs_off+0xd/0x30
[ 1235.678086] ? exc_page_fault+0xda/0xf0
[ 1235.683189] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.689658] RIP: 0033:0x7f5f84ee422e
[ 1235.694467] RSP: 002b:00007ffef5296d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.703759] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5f84ee422e
[ 1235.712569] RDX: 0000000000000001 RSI: 00005556e9ea0e10 RDI: 0000000000000007
[ 1235.721379] RBP: 00007ffef5296d40 R08: 0000000000000000 R09: 0000000000000000
[ 1235.730187] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5f84d5c390
[ 1235.738997] R13: 00005556e9ea0e10 R14: 0000000000000007 R15: 0000000000000003
[ 1235.747811]
[ 1235.751097] INFO: task python3:5717 is blocked on a mutex likely owned by task python3:3628.
[ 1235.761369] task:python3 state:D stack:25672 pid:3628 tgid:3628 ppid:3627 task_flags:0x400100 flags:0x00080802
[ 1235.774571] Call Trace:
[ 1235.778108]
[ 1235.781257] __schedule+0xcba/0x1940
[ 1235.786070] ? io_schedule_timeout+0x140/0x140
[ 1235.791854] ? find_held_lock+0x2b/0x80
[ 1235.796958] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.803135] schedule+0xd4/0x260
[ 1235.807554] schedule_timeout+0x13f/0x290
[ 1235.812852] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.819221] ? trace_contention_end+0xb3/0x1a0
[ 1235.825007] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.830699] ? ww_mutex_lock+0x160/0x160
[ 1235.835899] msleep+0x1b/0x30
[ 1235.840024] napi_set_threaded+0x3a5/0x440
[ 1235.845422] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.850909] netif_set_threaded+0x257/0x400
[ 1235.856403] threaded_store+0x1b8/0x220
[ 1235.861504] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.866800] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.872095] ? sysfs_kf_write+0x65/0x160
[ 1235.877294] ? sysfs_file_ops+0x140/0x140
[ 1235.882591] kernfs_fop_write_iter+0x335/0x510
[ 1235.888378] ? kernfs_ops+0xf0/0xf0
[ 1235.893087] new_sync_write+0x333/0x750
[ 1235.898191] ? __lock_acquire+0x508/0xc10
[ 1235.903487] ? new_sync_read+0x740/0x740
[ 1235.908687] ? lock_acquire.part.0+0xbc/0x260
[ 1235.914373] ? ksys_write+0x116/0x250
[ 1235.919283] vfs_write+0x657/0x9b0
[ 1235.923900] ksys_write+0x116/0x250
[ 1235.928614] ? __ia32_sys_read+0xc0/0xc0
[ 1235.933814] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.939403] ? rcu_is_watching+0x15/0xd0
[ 1235.944596] do_syscall_64+0xf3/0x650
[ 1235.949504] ? irq_exit_rcu+0x1a/0x30
[ 1235.954410] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.960878] RIP: 0033:0x7f3fe559522e
[ 1235.965685] RSP: 002b:00007ffdbaf6bd50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.974982] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3fe559522e
[ 1235.983790] RDX: 0000000000000001 RSI: 000055fe6c348e10 RDI: 0000000000000007
[ 1235.992601] RBP: 00007ffdbaf6bd60 R08: 0000000000000000 R09: 0000000000000000
[ 1236.001411] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3fe540d390
[ 1236.010217] R13: 000055fe6c348e10 R14: 0000000000000007 R15: 0000000000000003
[ 1236.019030]
[ 1236.022295] \x0aShowing all locks held in the system:
[ 1236.030835] 1 lock held by khungtaskd/185:
[ 1236.036231] #0: ffffffffa0efdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.047705] 2 locks held by systemd-journal/804:
[ 1236.053676] #0: ff1100180dd372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1236.063566] #1: ffffffffa1103120 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.076391] 4 locks held by python3/3628:
[ 1236.081685] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.091575] #1: ff1100015474c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.102528] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.113687] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.123783] 4 locks held by python3/5717:
[ 1236.129082] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.138970] #1: ff110001dce38880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.149933] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.161090] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.171177]
[ 1236.173642] =============================================\x0a
[ 1358.352578] INFO: task python3:5717 blocked for more than 491 seconds.
[ 1358.360726] Not tainted 7.1.0-rc1-qxpi-gf298010a814b #1
[ 1358.367980] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.377573] task:python3 state:D stack:25672 pid:5717 tgid:5717 ppid:5716 task_flags:0x400100 flags:0x00080002
[ 1358.390789] Call Trace:
[ 1358.394333]
[ 1358.397482] __schedule+0xcba/0x1940
[ 1358.402303] ? io_schedule_timeout+0x140/0x140
[ 1358.408092] ? find_held_lock+0x2b/0x80
[ 1358.413201] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.419095] schedule+0xd4/0x260
[ 1358.423520] schedule_preempt_disabled+0x18/0x30
[ 1358.429504] __mutex_lock+0x6a3/0x2000
[ 1358.434513] ? threaded_store+0x132/0x220
[ 1358.439820] ? ww_mutex_lock+0x160/0x160
[ 1358.445024] ? selinux_inode_free_security+0x330/0x330
[ 1358.451601] ? cap_capable+0x1d6/0x3e0
[ 1358.456614] ? threaded_store+0x132/0x220
[ 1358.461916] threaded_store+0x132/0x220
[ 1358.467023] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.472327] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.477622] ? sysfs_kf_write+0x65/0x160
[ 1358.482827] ? sysfs_file_ops+0x140/0x140
[ 1358.488127] kernfs_fop_write_iter+0x335/0x510
[ 1358.493916] ? kernfs_ops+0xf0/0xf0
[ 1358.498630] new_sync_write+0x333/0x750
[ 1358.503738] ? __lock_acquire+0x508/0xc10
[ 1358.509037] ? new_sync_read+0x740/0x740
[ 1358.514237] ? lock_acquire.part.0+0xbc/0x260
[ 1358.519927] ? ksys_write+0x116/0x250
[ 1358.524847] vfs_write+0x657/0x9b0
[ 1358.529464] ksys_write+0x116/0x250
[ 1358.534173] ? __ia32_sys_read+0xc0/0xc0
[ 1358.539374] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.544969] ? rcu_is_watching+0x15/0xd0
[ 1358.550177] do_syscall_64+0xf3/0x650
[ 1358.555086] ? trace_hardirqs_off+0xd/0x30
[ 1358.560486] ? exc_page_fault+0xda/0xf0
[ 1358.565595] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.572069] RIP: 0033:0x7f5f84ee422e
[ 1358.576884] RSP: 002b:00007ffef5296d30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.586187] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5f84ee422e
[ 1358.595000] RDX: 0000000000000001 RSI: 00005556e9ea0e10 RDI: 0000000000000007
[ 1358.603812] RBP: 00007ffef5296d40 R08: 0000000000000000 R09: 0000000000000000
[ 1358.612629] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5f84d5c390
[ 1358.621440] R13: 00005556e9ea0e10 R14: 0000000000000007 R15: 0000000000000003
[ 1358.630266]
[ 1358.633554] INFO: task python3:5717 is blocked on a mutex likely owned by task python3:3628.
[ 1358.643832] task:python3 state:D stack:25672 pid:3628 tgid:3628 ppid:3627 task_flags:0x400100 flags:0x00080802
[ 1358.657040] Call Trace:
[ 1358.660583]
[ 1358.663737] __schedule+0xcba/0x1940
[ 1358.668556] ? io_schedule_timeout+0x140/0x140
[ 1358.674346] ? find_held_lock+0x2b/0x80
[ 1358.679456] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.685627] schedule+0xd4/0x260
[ 1358.690048] schedule_timeout+0x13f/0x290
[ 1358.695351] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.701722] ? trace_contention_end+0xb3/0x1a0
[ 1358.707512] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.713210] ? ww_mutex_lock+0x160/0x160
[ 1358.718420] msleep+0x1b/0x30
[ 1358.722548] napi_set_threaded+0x3a5/0x440
[ 1358.727947] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.733450] netif_set_threaded+0x257/0x400
[ 1358.738956] threaded_store+0x1b8/0x220
[ 1358.744065] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.749366] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.754667] ? sysfs_kf_write+0x65/0x160
[ 1358.759871] ? sysfs_file_ops+0x140/0x140
[ 1358.765172] kernfs_fop_write_iter+0x335/0x510
[ 1358.770962] ? kernfs_ops+0xf0/0xf0
[ 1358.775679] new_sync_write+0x333/0x750
[ 1358.780786] ? __lock_acquire+0x508/0xc10
[ 1358.786087] ? new_sync_read+0x740/0x740
[ 1358.791292] ? lock_acquire.part.0+0xbc/0x260
[ 1358.796982] ? ksys_write+0x116/0x250
[ 1358.801903] vfs_write+0x657/0x9b0
[ 1358.806524] ksys_write+0x116/0x250
[ 1358.811240] ? __ia32_sys_read+0xc0/0xc0
[ 1358.816443] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.822037] ? rcu_is_watching+0x15/0xd0
[ 1358.827244] do_syscall_64+0xf3/0x650
[ 1358.832154] ? irq_exit_rcu+0x1a/0x30
[ 1358.837064] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.843536] RIP: 0033:0x7f3fe559522e
[ 1358.848347] RSP: 002b:00007ffdbaf6bd50 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.857649] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3fe559522e
[ 1358.866465] RDX: 0000000000000001 RSI: 000055fe6c348e10 RDI: 0000000000000007
[ 1358.875277] RBP: 00007ffdbaf6bd60 R08: 0000000000000000 R09: 0000000000000000
[ 1358.884091] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3fe540d390
[ 1358.892902] R13: 000055fe6c348e10 R14: 0000000000000007 R15: 0000000000000003
[ 1358.901727]
[ 1358.905001] \x0aShowing all locks held in the system:
[ 1358.913550] 1 lock held by khungtaskd/185:
[ 1358.918946] #0: ffffffffa0efdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.930430] 3 locks held by systemd-journal/804:
[ 1358.936410] #0: ff1100180e1b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1358.946310] #1: ffffffffa0efdee0 (rcu_read_lock){....}-{1:3}, at: __update_idle_core+0x5a/0x700
[ 1358.956988] #2: ff110001a4b309c0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.968549] 4 locks held by python3/3628:
[ 1358.973848] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.983746] #1: ff1100015474c480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.994719] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.005875] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.015980] 4 locks held by python3/5717:
[ 1359.021279] #0: ff1100012f8713e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.031174] #1: ff110001dce38880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.042145] #2: ff11000110b9eb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.053309] #3: ff11000110bf1d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.063410]
[ 1359.065878] =============================================\x0a