[ 762.872256] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 762.874747] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 763.004875] kselftest: Running tests in drivers/net
[ 989.699491] INFO: task python3:5426 blocked for more than 122 seconds.
[ 989.707730] Not tainted 7.1.0-rc6-cbqh-g596bda603a38 #1
[ 989.715685] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.725275] task:python3 state:D stack:25672 pid:5426 tgid:5426 ppid:5425 task_flags:0x400100 flags:0x00080000
[ 989.738483] Call Trace:
[ 989.742024]
[ 989.745174] __schedule+0xcc3/0x19c0
[ 989.749992] ? io_schedule_timeout+0x140/0x140
[ 989.755780] ? find_held_lock+0x2b/0x80
[ 989.760886] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.766771] schedule+0xd8/0x260
[ 989.771189] schedule_preempt_disabled+0x18/0x30
[ 989.777167] __mutex_lock+0x6af/0x2010
[ 989.782175] ? threaded_store+0x132/0x220
[ 989.787473] ? ww_mutex_lock+0x160/0x160
[ 989.792675] ? selinux_inode_free_security+0x330/0x330
[ 989.799245] ? cap_capable+0x1d6/0x3e0
[ 989.804255] ? threaded_store+0x132/0x220
[ 989.809552] threaded_store+0x132/0x220
[ 989.814655] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.819945] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.825245] ? sysfs_kf_write+0x65/0x160
[ 989.830448] ? sysfs_file_ops+0x140/0x140
[ 989.835745] kernfs_fop_write_iter+0x335/0x510
[ 989.841530] ? kernfs_ops+0xf0/0xf0
[ 989.846240] new_sync_write+0x333/0x750
[ 989.851345] ? __lock_acquire+0x508/0xc10
[ 989.856644] ? new_sync_read+0x740/0x740
[ 989.861847] ? lock_acquire.part.0+0xbc/0x260
[ 989.867535] ? ksys_write+0x116/0x250
[ 989.872447] vfs_write+0x657/0x9b0
[ 989.877061] ksys_write+0x116/0x250
[ 989.881776] ? __ia32_sys_read+0xc0/0xc0
[ 989.886975] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.892568] ? rcu_is_watching+0x15/0xd0
[ 989.897770] do_syscall_64+0xf3/0x650
[ 989.902677] ? trace_hardirqs_off+0xd/0x30
[ 989.908074] ? exc_page_fault+0xda/0xf0
[ 989.913177] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.919643] RIP: 0033:0x7f365c56764e
[ 989.924455] RSP: 002b:00007ffc2f200e80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.933755] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f365c56764e
[ 989.942562] RDX: 0000000000000001 RSI: 0000563218227420 RDI: 0000000000000007
[ 989.951371] RBP: 00007ffc2f200e90 R08: 0000000000000000 R09: 0000000000000000
[ 989.960181] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f365c3e0390
[ 989.968990] R13: 0000563218227420 R14: 0000000000000007 R15: 00007f365c375120
[ 989.977811]
[ 989.981102] INFO: task python3:5426 is blocked on a mutex likely owned by task python3:3391.
[ 989.991373] task:python3 state:D stack:25632 pid:3391 tgid:3391 ppid:3390 task_flags:0x400100 flags:0x00080802
[ 990.004578] Call Trace:
[ 990.008117]
[ 990.011268] __schedule+0xcc3/0x19c0
[ 990.016073] ? io_schedule_timeout+0x140/0x140
[ 990.021858] ? find_held_lock+0x2b/0x80
[ 990.026963] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.033140] schedule+0xd8/0x260
[ 990.037549] schedule_timeout+0x13f/0x290
[ 990.042847] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.049218] ? trace_contention_end+0xb3/0x1a0
[ 990.054996] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.060688] ? ww_mutex_lock+0x160/0x160
[ 990.065889] msleep+0x1b/0x30
[ 990.070016] napi_set_threaded+0x3a5/0x440
[ 990.075414] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.080907] netif_set_threaded+0x257/0x400
[ 990.086403] threaded_store+0x1b8/0x220
[ 990.091505] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.096803] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.102102] ? sysfs_kf_write+0x65/0x160
[ 990.107301] ? sysfs_file_ops+0x140/0x140
[ 990.112599] kernfs_fop_write_iter+0x335/0x510
[ 990.118378] ? kernfs_ops+0xf0/0xf0
[ 990.123090] new_sync_write+0x333/0x750
[ 990.128192] ? __lock_acquire+0x508/0xc10
[ 990.133491] ? new_sync_read+0x740/0x740
[ 990.138689] ? lock_acquire.part.0+0xbc/0x260
[ 990.144376] ? ksys_write+0x116/0x250
[ 990.149285] vfs_write+0x657/0x9b0
[ 990.153904] ksys_write+0x116/0x250
[ 990.158617] ? __ia32_sys_read+0xc0/0xc0
[ 990.163815] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.169403] ? rcu_is_watching+0x15/0xd0
[ 990.174602] do_syscall_64+0xf3/0x650
[ 990.179510] ? irq_exit_rcu+0x1a/0x30
[ 990.184418] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.190878] RIP: 0033:0x7fe61fac864e
[ 990.195688] RSP: 002b:00007ffc51350590 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.204975] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe61fac864e
[ 990.213782] RDX: 0000000000000001 RSI: 0000557965ce1420 RDI: 0000000000000007
[ 990.222589] RBP: 00007ffc513505a0 R08: 0000000000000000 R09: 0000000000000000
[ 990.231390] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe61f941390
[ 990.240200] R13: 0000557965ce1420 R14: 0000000000000007 R15: 00007fe61f8d5120
[ 990.249014]
[ 990.252277] \x0aShowing all locks held in the system:
[ 990.260814] 1 lock held by khungtaskd/184:
[ 990.266208] #0: ffffffffb52fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.277693] 4 locks held by python3/3391:
[ 990.282990] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.292879] #1: ff1100013f4ab280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.303841] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.314988] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.325080] 4 locks held by python3/5426:
[ 990.330377] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.340266] #1: ff110001a9a72680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.351220] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.362379] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.372474]
[ 990.374943] =============================================\x0a
[ 1112.580666] INFO: task python3:5426 blocked for more than 245 seconds.
[ 1112.588817] Not tainted 7.1.0-rc6-cbqh-g596bda603a38 #1
[ 1112.596069] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.605658] task:python3 state:D stack:25672 pid:5426 tgid:5426 ppid:5425 task_flags:0x400100 flags:0x00080000
[ 1112.618866] Call Trace:
[ 1112.622407]
[ 1112.625557] __schedule+0xcc3/0x19c0
[ 1112.630373] ? io_schedule_timeout+0x140/0x140
[ 1112.636160] ? find_held_lock+0x2b/0x80
[ 1112.641264] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.647150] schedule+0xd8/0x260
[ 1112.651568] schedule_preempt_disabled+0x18/0x30
[ 1112.657547] __mutex_lock+0x6af/0x2010
[ 1112.662554] ? threaded_store+0x132/0x220
[ 1112.667854] ? ww_mutex_lock+0x160/0x160
[ 1112.673054] ? selinux_inode_free_security+0x330/0x330
[ 1112.679625] ? cap_capable+0x1d6/0x3e0
[ 1112.684634] ? threaded_store+0x132/0x220
[ 1112.689935] threaded_store+0x132/0x220
[ 1112.695038] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.700329] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.705626] ? sysfs_kf_write+0x65/0x160
[ 1112.710825] ? sysfs_file_ops+0x140/0x140
[ 1112.716121] kernfs_fop_write_iter+0x335/0x510
[ 1112.721909] ? kernfs_ops+0xf0/0xf0
[ 1112.726621] new_sync_write+0x333/0x750
[ 1112.731727] ? __lock_acquire+0x508/0xc10
[ 1112.737023] ? new_sync_read+0x740/0x740
[ 1112.742227] ? lock_acquire.part.0+0xbc/0x260
[ 1112.747915] ? ksys_write+0x116/0x250
[ 1112.752825] vfs_write+0x657/0x9b0
[ 1112.757443] ksys_write+0x116/0x250
[ 1112.762157] ? __ia32_sys_read+0xc0/0xc0
[ 1112.767358] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.772949] ? rcu_is_watching+0x15/0xd0
[ 1112.778151] do_syscall_64+0xf3/0x650
[ 1112.783056] ? trace_hardirqs_off+0xd/0x30
[ 1112.788451] ? exc_page_fault+0xda/0xf0
[ 1112.793556] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.800026] RIP: 0033:0x7f365c56764e
[ 1112.804836] RSP: 002b:00007ffc2f200e80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.814136] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f365c56764e
[ 1112.822943] RDX: 0000000000000001 RSI: 0000563218227420 RDI: 0000000000000007
[ 1112.831753] RBP: 00007ffc2f200e90 R08: 0000000000000000 R09: 0000000000000000
[ 1112.840563] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f365c3e0390
[ 1112.849369] R13: 0000563218227420 R14: 0000000000000007 R15: 00007f365c375120
[ 1112.858182]
[ 1112.861472] INFO: task python3:5426 is blocked on a mutex likely owned by task python3:3391.
[ 1112.871744] task:python3 state:D stack:25632 pid:3391 tgid:3391 ppid:3390 task_flags:0x400100 flags:0x00080802
[ 1112.884948] Call Trace:
[ 1112.888486]
[ 1112.891638] __schedule+0xcc3/0x19c0
[ 1112.896450] ? io_schedule_timeout+0x140/0x140
[ 1112.902236] ? find_held_lock+0x2b/0x80
[ 1112.907342] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.913520] schedule+0xd8/0x260
[ 1112.917937] schedule_timeout+0x13f/0x290
[ 1112.923235] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.929595] ? trace_contention_end+0xb3/0x1a0
[ 1112.935383] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.941074] ? ww_mutex_lock+0x160/0x160
[ 1112.946274] msleep+0x1b/0x30
[ 1112.950400] napi_set_threaded+0x3a5/0x440
[ 1112.955795] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.961288] netif_set_threaded+0x257/0x400
[ 1112.966781] threaded_store+0x1b8/0x220
[ 1112.971881] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.977181] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.982479] ? sysfs_kf_write+0x65/0x160
[ 1112.987680] ? sysfs_file_ops+0x140/0x140
[ 1112.992977] kernfs_fop_write_iter+0x335/0x510
[ 1112.998764] ? kernfs_ops+0xf0/0xf0
[ 1113.003475] new_sync_write+0x333/0x750
[ 1113.008579] ? __lock_acquire+0x508/0xc10
[ 1113.013873] ? new_sync_read+0x740/0x740
[ 1113.019076] ? lock_acquire.part.0+0xbc/0x260
[ 1113.024761] ? ksys_write+0x116/0x250
[ 1113.029670] vfs_write+0x657/0x9b0
[ 1113.034287] ksys_write+0x116/0x250
[ 1113.039000] ? __ia32_sys_read+0xc0/0xc0
[ 1113.044197] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.049787] ? rcu_is_watching+0x15/0xd0
[ 1113.054990] do_syscall_64+0xf3/0x650
[ 1113.059895] ? irq_exit_rcu+0x1a/0x30
[ 1113.064802] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.071270] RIP: 0033:0x7fe61fac864e
[ 1113.076078] RSP: 002b:00007ffc51350590 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.085373] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe61fac864e
[ 1113.094180] RDX: 0000000000000001 RSI: 0000557965ce1420 RDI: 0000000000000007
[ 1113.102991] RBP: 00007ffc513505a0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.111800] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe61f941390
[ 1113.120609] R13: 0000557965ce1420 R14: 0000000000000007 R15: 00007fe61f8d5120
[ 1113.129424]
[ 1113.132688] \x0aShowing all locks held in the system:
[ 1113.141227] 1 lock held by khungtaskd/184:
[ 1113.146622] #0: ffffffffb52fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.158106] 4 locks held by python3/3391:
[ 1113.163393] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.173284] #1: ff1100013f4ab280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.184247] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.195404] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.205507] 4 locks held by python3/5426:
[ 1113.210801] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.220692] #1: ff110001a9a72680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.231653] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.242803] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.252894]
[ 1113.255362] =============================================\x0a
[ 1235.461794] INFO: task python3:5426 blocked for more than 368 seconds.
[ 1235.469944] Not tainted 7.1.0-rc6-cbqh-g596bda603a38 #1
[ 1235.477197] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.486784] task:python3 state:D stack:25672 pid:5426 tgid:5426 ppid:5425 task_flags:0x400100 flags:0x00080002
[ 1235.499993] Call Trace:
[ 1235.503534]
[ 1235.503539] __schedule+0xcc3/0x19c0
[ 1235.503550] ? io_schedule_timeout+0x140/0x140
[ 1235.503554] ? find_held_lock+0x2b/0x80
[ 1235.522455] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.528337] schedule+0xd8/0x260
[ 1235.532753] schedule_preempt_disabled+0x18/0x30
[ 1235.538733] __mutex_lock+0x6af/0x2010
[ 1235.543735] ? threaded_store+0x132/0x220
[ 1235.549032] ? ww_mutex_lock+0x160/0x160
[ 1235.554229] ? selinux_inode_free_security+0x330/0x330
[ 1235.560798] ? cap_capable+0x1d6/0x3e0
[ 1235.565805] ? threaded_store+0x132/0x220
[ 1235.571103] threaded_store+0x132/0x220
[ 1235.576195] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.581493] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.586789] ? sysfs_kf_write+0x65/0x160
[ 1235.591989] ? sysfs_file_ops+0x140/0x140
[ 1235.597276] kernfs_fop_write_iter+0x335/0x510
[ 1235.603058] ? kernfs_ops+0xf0/0xf0
[ 1235.607768] new_sync_write+0x333/0x750
[ 1235.612872] ? __lock_acquire+0x508/0xc10
[ 1235.618166] ? new_sync_read+0x740/0x740
[ 1235.623368] ? lock_acquire.part.0+0xbc/0x260
[ 1235.629053] ? ksys_write+0x116/0x250
[ 1235.633961] vfs_write+0x657/0x9b0
[ 1235.638573] ksys_write+0x116/0x250
[ 1235.643285] ? __ia32_sys_read+0xc0/0xc0
[ 1235.648482] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.654072] ? rcu_is_watching+0x15/0xd0
[ 1235.659272] do_syscall_64+0xf3/0x650
[ 1235.664175] ? trace_hardirqs_off+0xd/0x30
[ 1235.669570] ? exc_page_fault+0xda/0xf0
[ 1235.674671] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.681136] RIP: 0033:0x7f365c56764e
[ 1235.685945] RSP: 002b:00007ffc2f200e80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.695245] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f365c56764e
[ 1235.704050] RDX: 0000000000000001 RSI: 0000563218227420 RDI: 0000000000000007
[ 1235.712858] RBP: 00007ffc2f200e90 R08: 0000000000000000 R09: 0000000000000000
[ 1235.721658] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f365c3e0390
[ 1235.730467] R13: 0000563218227420 R14: 0000000000000007 R15: 00007f365c375120
[ 1235.739280]
[ 1235.742611] INFO: task python3:5426 is blocked on a mutex likely owned by task python3:3391.
[ 1235.752881] task:python3 state:D stack:25632 pid:3391 tgid:3391 ppid:3390 task_flags:0x400100 flags:0x00080802
[ 1235.766084] Call Trace:
[ 1235.769620]
[ 1235.772771] __schedule+0xcc3/0x19c0
[ 1235.777582] ? io_schedule_timeout+0x140/0x140
[ 1235.783365] ? find_held_lock+0x2b/0x80
[ 1235.788466] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.794633] schedule+0xd8/0x260
[ 1235.799051] schedule_timeout+0x13f/0x290
[ 1235.804346] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.810714] ? trace_contention_end+0xb3/0x1a0
[ 1235.816496] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.822187] ? ww_mutex_lock+0x160/0x160
[ 1235.827388] msleep+0x1b/0x30
[ 1235.831512] napi_set_threaded+0x3a5/0x440
[ 1235.836907] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.842391] netif_set_threaded+0x257/0x400
[ 1235.847883] threaded_store+0x1b8/0x220
[ 1235.852975] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.858269] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.863563] ? sysfs_kf_write+0x65/0x160
[ 1235.868762] ? sysfs_file_ops+0x140/0x140
[ 1235.874058] kernfs_fop_write_iter+0x335/0x510
[ 1235.879841] ? kernfs_ops+0xf0/0xf0
[ 1235.884552] new_sync_write+0x333/0x750
[ 1235.889654] ? __lock_acquire+0x508/0xc10
[ 1235.894950] ? new_sync_read+0x740/0x740
[ 1235.900147] ? lock_acquire.part.0+0xbc/0x260
[ 1235.905833] ? ksys_write+0x116/0x250
[ 1235.910742] vfs_write+0x657/0x9b0
[ 1235.915355] ksys_write+0x116/0x250
[ 1235.920068] ? __ia32_sys_read+0xc0/0xc0
[ 1235.925265] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.930853] ? rcu_is_watching+0x15/0xd0
[ 1235.936052] do_syscall_64+0xf3/0x650
[ 1235.940957] ? irq_exit_rcu+0x1a/0x30
[ 1235.945863] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.952329] RIP: 0033:0x7fe61fac864e
[ 1235.957136] RSP: 002b:00007ffc51350590 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.966433] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe61fac864e
[ 1235.975246] RDX: 0000000000000001 RSI: 0000557965ce1420 RDI: 0000000000000007
[ 1235.984055] RBP: 00007ffc513505a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.992859] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe61f941390
[ 1236.001663] R13: 0000557965ce1420 R14: 0000000000000007 R15: 00007fe61f8d5120
[ 1236.010474]
[ 1236.013739] \x0aShowing all locks held in the system:
[ 1236.022286] 1 lock held by khungtaskd/184:
[ 1236.027678] #0: ffffffffb52fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.039165] 4 locks held by python3/3391:
[ 1236.044461] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.054352] #1: ff1100013f4ab280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.065312] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.076467] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.086575] 4 locks held by python3/5426:
[ 1236.091870] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.101756] #1: ff110001a9a72680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.112715] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.123872] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.133959] 5 locks held by (systemd-user-)/6906:
[ 1236.140035] #0: ff110001314ec3e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.149921] #1: ff110001a30ce280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.160883] #2: ffffffffb5369c20 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1236.171549] #3: ffffffffb51687b0 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1236.183190] #4: ffffffffb5369990 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1236.195615]
[ 1236.198080] =============================================\x0a
[ 1358.342936] INFO: task python3:5426 blocked for more than 491 seconds.
[ 1358.351076] Not tainted 7.1.0-rc6-cbqh-g596bda603a38 #1
[ 1358.358324] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.367916] task:python3 state:D stack:25672 pid:5426 tgid:5426 ppid:5425 task_flags:0x400100 flags:0x00080002
[ 1358.381121] Call Trace:
[ 1358.384660]
[ 1358.387811] __schedule+0xcc3/0x19c0
[ 1358.392627] ? io_schedule_timeout+0x140/0x140
[ 1358.398411] ? find_held_lock+0x2b/0x80
[ 1358.403515] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.409401] schedule+0xd8/0x260
[ 1358.413821] schedule_preempt_disabled+0x18/0x30
[ 1358.419791] __mutex_lock+0x6af/0x2010
[ 1358.424796] ? threaded_store+0x132/0x220
[ 1358.430097] ? ww_mutex_lock+0x160/0x160
[ 1358.435297] ? selinux_inode_free_security+0x330/0x330
[ 1358.441869] ? cap_capable+0x1d6/0x3e0
[ 1358.446879] ? threaded_store+0x132/0x220
[ 1358.452175] threaded_store+0x132/0x220
[ 1358.457278] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.462576] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.467876] ? sysfs_kf_write+0x65/0x160
[ 1358.473075] ? sysfs_file_ops+0x140/0x140
[ 1358.478369] kernfs_fop_write_iter+0x335/0x510
[ 1358.484154] ? kernfs_ops+0xf0/0xf0
[ 1358.488866] new_sync_write+0x333/0x750
[ 1358.493972] ? __lock_acquire+0x508/0xc10
[ 1358.499270] ? new_sync_read+0x740/0x740
[ 1358.504470] ? lock_acquire.part.0+0xbc/0x260
[ 1358.510155] ? ksys_write+0x116/0x250
[ 1358.515065] vfs_write+0x657/0x9b0
[ 1358.519679] ksys_write+0x116/0x250
[ 1358.524390] ? __ia32_sys_read+0xc0/0xc0
[ 1358.529588] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.535180] ? rcu_is_watching+0x15/0xd0
[ 1358.540383] do_syscall_64+0xf3/0x650
[ 1358.545289] ? trace_hardirqs_off+0xd/0x30
[ 1358.550687] ? exc_page_fault+0xda/0xf0
[ 1358.555793] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.562260] RIP: 0033:0x7f365c56764e
[ 1358.567068] RSP: 002b:00007ffc2f200e80 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.576365] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f365c56764e
[ 1358.585164] RDX: 0000000000000001 RSI: 0000563218227420 RDI: 0000000000000007
[ 1358.593971] RBP: 00007ffc2f200e90 R08: 0000000000000000 R09: 0000000000000000
[ 1358.602786] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f365c3e0390
[ 1358.611594] R13: 0000563218227420 R14: 0000000000000007 R15: 00007f365c375120
[ 1358.620409]
[ 1358.623710] INFO: task python3:5426 is blocked on a mutex likely owned by task python3:3391.
[ 1358.633982] task:python3 state:D stack:25632 pid:3391 tgid:3391 ppid:3390 task_flags:0x400100 flags:0x00080802
[ 1358.647188] Call Trace:
[ 1358.650726]
[ 1358.653875] __schedule+0xcc3/0x19c0
[ 1358.658689] ? io_schedule_timeout+0x140/0x140
[ 1358.664475] ? find_held_lock+0x2b/0x80
[ 1358.669581] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.675759] schedule+0xd8/0x260
[ 1358.680179] schedule_timeout+0x13f/0x290
[ 1358.685467] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.691838] ? trace_contention_end+0xb3/0x1a0
[ 1358.697622] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.703314] ? ww_mutex_lock+0x160/0x160
[ 1358.708516] msleep+0x1b/0x30
[ 1358.712643] napi_set_threaded+0x3a5/0x440
[ 1358.718038] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.723533] netif_set_threaded+0x257/0x400
[ 1358.729026] threaded_store+0x1b8/0x220
[ 1358.734129] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.739428] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.744724] ? sysfs_kf_write+0x65/0x160
[ 1358.749922] ? sysfs_file_ops+0x140/0x140
[ 1358.755219] kernfs_fop_write_iter+0x335/0x510
[ 1358.761002] ? kernfs_ops+0xf0/0xf0
[ 1358.765714] new_sync_write+0x333/0x750
[ 1358.770810] ? __lock_acquire+0x508/0xc10
[ 1358.776106] ? new_sync_read+0x740/0x740
[ 1358.781308] ? lock_acquire.part.0+0xbc/0x260
[ 1358.786995] ? ksys_write+0x116/0x250
[ 1358.791907] vfs_write+0x657/0x9b0
[ 1358.796524] ksys_write+0x116/0x250
[ 1358.801239] ? __ia32_sys_read+0xc0/0xc0
[ 1358.806437] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.812029] ? rcu_is_watching+0x15/0xd0
[ 1358.817227] do_syscall_64+0xf3/0x650
[ 1358.822134] ? irq_exit_rcu+0x1a/0x30
[ 1358.827033] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.833498] RIP: 0033:0x7fe61fac864e
[ 1358.838308] RSP: 002b:00007ffc51350590 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.847596] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe61fac864e
[ 1358.856406] RDX: 0000000000000001 RSI: 0000557965ce1420 RDI: 0000000000000007
[ 1358.865213] RBP: 00007ffc513505a0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.874011] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe61f941390
[ 1358.882821] R13: 0000557965ce1420 R14: 0000000000000007 R15: 00007fe61f8d5120
[ 1358.891633]
[ 1358.894903] \x0aShowing all locks held in the system:
[ 1358.903445] 1 lock held by khungtaskd/184:
[ 1358.908829] #0: ffffffffb52fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.920304] 3 locks held by systemd-journal/801:
[ 1358.926276] #0: ff1100180e4b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.936167] #1: ffffffffb5502ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.948986] #2: ff110001a56a8b20 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.960542] 4 locks held by python3/3391:
[ 1358.965839] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.975730] #1: ff1100013f4ab280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.986692] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.997850] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.007953] 4 locks held by python3/5426:
[ 1359.013248] #0: ff110001313e43e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.023139] #1: ff110001a9a72680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.034100] #2: ff11000123638b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.045257] #3: ff110001235c1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.055345]
[ 1359.057813] =============================================\x0a