[ 763.589668] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 763.593070] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 763.725137] kselftest: Running tests in drivers/net
[ 989.684145] INFO: task python3:5422 blocked for more than 122 seconds.
[ 989.692381] Not tainted 7.1.0-rc7-ccpn-gb37f439f8a5a #1
[ 989.700281] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.709870] task:python3 state:D stack:25552 pid:5422 tgid:5422 ppid:5421 task_flags:0x400100 flags:0x00080000
[ 989.723079] Call Trace:
[ 989.726620]
[ 989.729770] __schedule+0xcc3/0x19c0
[ 989.734595] ? io_schedule_timeout+0x140/0x140
[ 989.740394] ? find_held_lock+0x2b/0x80
[ 989.745503] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.751389] schedule+0xd8/0x260
[ 989.755807] schedule_preempt_disabled+0x18/0x30
[ 989.761788] __mutex_lock+0x6af/0x2010
[ 989.766793] ? threaded_store+0x132/0x220
[ 989.772091] ? ww_mutex_lock+0x160/0x160
[ 989.777291] ? selinux_inode_free_security+0x330/0x330
[ 989.783862] ? cap_capable+0x1d6/0x3e0
[ 989.788870] ? threaded_store+0x132/0x220
[ 989.794168] threaded_store+0x132/0x220
[ 989.799271] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.804559] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.809857] ? sysfs_kf_write+0x65/0x160
[ 989.815058] ? sysfs_file_ops+0x140/0x140
[ 989.820352] kernfs_fop_write_iter+0x335/0x510
[ 989.826136] ? kernfs_ops+0xf0/0xf0
[ 989.830847] new_sync_write+0x333/0x750
[ 989.835953] ? __lock_acquire+0x508/0xc10
[ 989.841249] ? new_sync_read+0x740/0x740
[ 989.846448] ? lock_acquire.part.0+0xbc/0x260
[ 989.852137] ? ksys_write+0x116/0x250
[ 989.857046] vfs_write+0x657/0x9b0
[ 989.861660] ksys_write+0x116/0x250
[ 989.866373] ? __ia32_sys_read+0xc0/0xc0
[ 989.871562] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.877150] ? rcu_is_watching+0x15/0xd0
[ 989.882344] do_syscall_64+0xf3/0x650
[ 989.887249] ? trace_hardirqs_off+0xd/0x30
[ 989.892644] ? exc_page_fault+0xda/0xf0
[ 989.897747] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.904215] RIP: 0033:0x7f325ee1a64e
[ 989.909036] RSP: 002b:00007ffc2a1d61b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.918335] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f325ee1a64e
[ 989.927144] RDX: 0000000000000001 RSI: 0000561991474bf0 RDI: 0000000000000007
[ 989.935953] RBP: 00007ffc2a1d61c0 R08: 0000000000000000 R09: 0000000000000000
[ 989.944763] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f325ec93390
[ 989.953572] R13: 0000561991474bf0 R14: 0000000000000007 R15: 00007f325ec29120
[ 989.962389]
[ 989.965703] INFO: task python3:5422 is blocked on a mutex likely owned by task python3:3432.
[ 989.975978] task:python3 state:D stack:25672 pid:3432 tgid:3432 ppid:3431 task_flags:0x400100 flags:0x00080802
[ 989.989174] Call Trace:
[ 989.992713]
[ 989.995864] __schedule+0xcc3/0x19c0
[ 990.000677] ? io_schedule_timeout+0x140/0x140
[ 990.006461] ? find_held_lock+0x2b/0x80
[ 990.011567] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.017744] schedule+0xd8/0x260
[ 990.022163] schedule_timeout+0x13f/0x290
[ 990.027458] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.033828] ? trace_contention_end+0xb3/0x1a0
[ 990.039614] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.045306] ? ww_mutex_lock+0x160/0x160
[ 990.050506] msleep+0x1b/0x30
[ 990.054631] napi_set_threaded+0x3a5/0x440
[ 990.060025] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.065518] netif_set_threaded+0x257/0x400
[ 990.071011] threaded_store+0x1b8/0x220
[ 990.076104] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.081401] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.086688] ? sysfs_kf_write+0x65/0x160
[ 990.091889] ? sysfs_file_ops+0x140/0x140
[ 990.097185] kernfs_fop_write_iter+0x335/0x510
[ 990.102969] ? kernfs_ops+0xf0/0xf0
[ 990.107681] new_sync_write+0x333/0x750
[ 990.112785] ? __lock_acquire+0x508/0xc10
[ 990.118081] ? new_sync_read+0x740/0x740
[ 990.123282] ? lock_acquire.part.0+0xbc/0x260
[ 990.128970] ? ksys_write+0x116/0x250
[ 990.133869] vfs_write+0x657/0x9b0
[ 990.138476] ksys_write+0x116/0x250
[ 990.143191] ? __ia32_sys_read+0xc0/0xc0
[ 990.148389] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.153980] ? rcu_is_watching+0x15/0xd0
[ 990.159182] do_syscall_64+0xf3/0x650
[ 990.164087] ? trace_hardirqs_off+0xd/0x30
[ 990.169484] ? exc_page_fault+0xda/0xf0
[ 990.174587] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.181055] RIP: 0033:0x7f45e703964e
[ 990.185866] RSP: 002b:00007ffea2670250 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.195164] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f45e703964e
[ 990.203974] RDX: 0000000000000001 RSI: 000055ec8716ebf0 RDI: 0000000000000007
[ 990.212785] RBP: 00007ffea2670260 R08: 0000000000000000 R09: 0000000000000000
[ 990.221594] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f45e6eb2390
[ 990.230403] R13: 000055ec8716ebf0 R14: 0000000000000007 R15: 00007f45e6e45120
[ 990.239216]
[ 990.242477] \x0aShowing all locks held in the system:
[ 990.251023] 1 lock held by khungtaskd/184:
[ 990.256415] #0: ffffffff934fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.267892] 2 locks held by systemd-journal/803:
[ 990.273871] #0: ff1100180e3b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.283758] #1: ffffffff93702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.296576] 4 locks held by python3/3432:
[ 990.301870] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.311758] #1: ff1100013d066280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.322721] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.333879] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.343979] 4 locks held by python3/5422:
[ 990.349277] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.359164] #1: ff110001865fac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.370128] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.381285] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.391374]
[ 990.393833] =============================================\x0a
[ 1112.565115] INFO: task python3:5422 blocked for more than 245 seconds.
[ 1112.573263] Not tainted 7.1.0-rc7-ccpn-gb37f439f8a5a #1
[ 1112.580513] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.590101] task:python3 state:D stack:25552 pid:5422 tgid:5422 ppid:5421 task_flags:0x400100 flags:0x00080000
[ 1112.603300] Call Trace:
[ 1112.606843]
[ 1112.609987] __schedule+0xcc3/0x19c0
[ 1112.614806] ? io_schedule_timeout+0x140/0x140
[ 1112.620592] ? find_held_lock+0x2b/0x80
[ 1112.625698] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.631585] schedule+0xd8/0x260
[ 1112.636006] schedule_preempt_disabled+0x18/0x30
[ 1112.641988] __mutex_lock+0x6af/0x2010
[ 1112.646993] ? threaded_store+0x132/0x220
[ 1112.652294] ? ww_mutex_lock+0x160/0x160
[ 1112.657494] ? selinux_inode_free_security+0x330/0x330
[ 1112.664063] ? cap_capable+0x1d6/0x3e0
[ 1112.669072] ? threaded_store+0x132/0x220
[ 1112.674369] threaded_store+0x132/0x220
[ 1112.679473] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.684769] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.690068] ? sysfs_kf_write+0x65/0x160
[ 1112.695271] ? sysfs_file_ops+0x140/0x140
[ 1112.700568] kernfs_fop_write_iter+0x335/0x510
[ 1112.706346] ? kernfs_ops+0xf0/0xf0
[ 1112.711059] new_sync_write+0x333/0x750
[ 1112.716164] ? __lock_acquire+0x508/0xc10
[ 1112.721451] ? new_sync_read+0x740/0x740
[ 1112.726653] ? lock_acquire.part.0+0xbc/0x260
[ 1112.732340] ? ksys_write+0x116/0x250
[ 1112.737248] vfs_write+0x657/0x9b0
[ 1112.741866] ksys_write+0x116/0x250
[ 1112.746578] ? __ia32_sys_read+0xc0/0xc0
[ 1112.751778] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.757369] ? rcu_is_watching+0x15/0xd0
[ 1112.762574] do_syscall_64+0xf3/0x650
[ 1112.767483] ? trace_hardirqs_off+0xd/0x30
[ 1112.772879] ? exc_page_fault+0xda/0xf0
[ 1112.777973] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.784434] RIP: 0033:0x7f325ee1a64e
[ 1112.789244] RSP: 002b:00007ffc2a1d61b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.798542] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f325ee1a64e
[ 1112.807351] RDX: 0000000000000001 RSI: 0000561991474bf0 RDI: 0000000000000007
[ 1112.816162] RBP: 00007ffc2a1d61c0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.824972] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f325ec93390
[ 1112.833780] R13: 0000561991474bf0 R14: 0000000000000007 R15: 00007f325ec29120
[ 1112.842584]
[ 1112.845871] INFO: task python3:5422 is blocked on a mutex likely owned by task python3:3432.
[ 1112.856142] task:python3 state:D stack:25672 pid:3432 tgid:3432 ppid:3431 task_flags:0x400100 flags:0x00080802
[ 1112.869344] Call Trace:
[ 1112.872881]
[ 1112.876033] __schedule+0xcc3/0x19c0
[ 1112.880846] ? io_schedule_timeout+0x140/0x140
[ 1112.886631] ? find_held_lock+0x2b/0x80
[ 1112.891735] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.897912] schedule+0xd8/0x260
[ 1112.902331] schedule_timeout+0x13f/0x290
[ 1112.907627] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.913997] ? trace_contention_end+0xb3/0x1a0
[ 1112.919783] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.925476] ? ww_mutex_lock+0x160/0x160
[ 1112.930676] msleep+0x1b/0x30
[ 1112.934801] napi_set_threaded+0x3a5/0x440
[ 1112.940200] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.945697] netif_set_threaded+0x257/0x400
[ 1112.951191] threaded_store+0x1b8/0x220
[ 1112.956296] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.961593] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.966888] ? sysfs_kf_write+0x65/0x160
[ 1112.972087] ? sysfs_file_ops+0x140/0x140
[ 1112.977383] kernfs_fop_write_iter+0x335/0x510
[ 1112.983170] ? kernfs_ops+0xf0/0xf0
[ 1112.987880] new_sync_write+0x333/0x750
[ 1112.992984] ? __lock_acquire+0x508/0xc10
[ 1112.998278] ? new_sync_read+0x740/0x740
[ 1113.003478] ? lock_acquire.part.0+0xbc/0x260
[ 1113.009156] ? ksys_write+0x116/0x250
[ 1113.014068] vfs_write+0x657/0x9b0
[ 1113.018684] ksys_write+0x116/0x250
[ 1113.023398] ? __ia32_sys_read+0xc0/0xc0
[ 1113.028600] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.034190] ? rcu_is_watching+0x15/0xd0
[ 1113.039384] do_syscall_64+0xf3/0x650
[ 1113.044292] ? trace_hardirqs_off+0xd/0x30
[ 1113.049686] ? exc_page_fault+0xda/0xf0
[ 1113.054787] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.061253] RIP: 0033:0x7f45e703964e
[ 1113.066063] RSP: 002b:00007ffea2670250 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.075358] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f45e703964e
[ 1113.084166] RDX: 0000000000000001 RSI: 000055ec8716ebf0 RDI: 0000000000000007
[ 1113.092971] RBP: 00007ffea2670260 R08: 0000000000000000 R09: 0000000000000000
[ 1113.101781] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f45e6eb2390
[ 1113.110588] R13: 000055ec8716ebf0 R14: 0000000000000007 R15: 00007f45e6e45120
[ 1113.119393]
[ 1113.122660] \x0aShowing all locks held in the system:
[ 1113.131197] 1 lock held by khungtaskd/184:
[ 1113.136590] #0: ffffffff934fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.148069] 4 locks held by python3/3432:
[ 1113.153366] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.163255] #1: ff1100013d066280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.174219] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.185374] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.195469] 4 locks held by python3/5422:
[ 1113.200765] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.210654] #1: ff110001865fac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.221615] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.232772] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.242863]
[ 1113.245332] =============================================\x0a
[ 1149.429423] kworker/u130:1 (228) used greatest stack depth: 22176 bytes left
[ 1235.446221] INFO: task python3:5422 blocked for more than 368 seconds.
[ 1235.454371] Not tainted 7.1.0-rc7-ccpn-gb37f439f8a5a #1
[ 1235.461621] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.471211] task:python3 state:D stack:25552 pid:5422 tgid:5422 ppid:5421 task_flags:0x400100 flags:0x00080002
[ 1235.484419] Call Trace:
[ 1235.487959]
[ 1235.491113] __schedule+0xcc3/0x19c0
[ 1235.495932] ? io_schedule_timeout+0x140/0x140
[ 1235.501719] ? find_held_lock+0x2b/0x80
[ 1235.506826] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.512710] schedule+0xd8/0x260
[ 1235.517131] schedule_preempt_disabled+0x18/0x30
[ 1235.523112] __mutex_lock+0x6af/0x2010
[ 1235.528120] ? threaded_store+0x132/0x220
[ 1235.533420] ? ww_mutex_lock+0x160/0x160
[ 1235.538620] ? selinux_inode_free_security+0x330/0x330
[ 1235.545189] ? cap_capable+0x1d6/0x3e0
[ 1235.550200] ? threaded_store+0x132/0x220
[ 1235.555498] threaded_store+0x132/0x220
[ 1235.560605] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.565900] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.571189] ? sysfs_kf_write+0x65/0x160
[ 1235.576389] ? sysfs_file_ops+0x140/0x140
[ 1235.581676] kernfs_fop_write_iter+0x335/0x510
[ 1235.587462] ? kernfs_ops+0xf0/0xf0
[ 1235.592164] new_sync_write+0x333/0x750
[ 1235.597267] ? __lock_acquire+0x508/0xc10
[ 1235.602562] ? new_sync_read+0x740/0x740
[ 1235.607763] ? lock_acquire.part.0+0xbc/0x260
[ 1235.613449] ? ksys_write+0x116/0x250
[ 1235.618358] vfs_write+0x657/0x9b0
[ 1235.622975] ksys_write+0x116/0x250
[ 1235.627687] ? __ia32_sys_read+0xc0/0xc0
[ 1235.632879] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.638471] ? rcu_is_watching+0x15/0xd0
[ 1235.643675] do_syscall_64+0xf3/0x650
[ 1235.648580] ? trace_hardirqs_off+0xd/0x30
[ 1235.653977] ? exc_page_fault+0xda/0xf0
[ 1235.659072] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.665541] RIP: 0033:0x7f325ee1a64e
[ 1235.670351] RSP: 002b:00007ffc2a1d61b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.679650] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f325ee1a64e
[ 1235.688459] RDX: 0000000000000001 RSI: 0000561991474bf0 RDI: 0000000000000007
[ 1235.697258] RBP: 00007ffc2a1d61c0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.706072] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f325ec93390
[ 1235.714881] R13: 0000561991474bf0 R14: 0000000000000007 R15: 00007f325ec29120
[ 1235.723695]
[ 1235.726995] INFO: task python3:5422 is blocked on a mutex likely owned by task python3:3432.
[ 1235.737272] task:python3 state:D stack:25672 pid:3432 tgid:3432 ppid:3431 task_flags:0x400100 flags:0x00080802
[ 1235.750474] Call Trace:
[ 1235.754011]
[ 1235.757165] __schedule+0xcc3/0x19c0
[ 1235.761977] ? io_schedule_timeout+0x140/0x140
[ 1235.767763] ? find_held_lock+0x2b/0x80
[ 1235.772858] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.779035] schedule+0xd8/0x260
[ 1235.783454] schedule_timeout+0x13f/0x290
[ 1235.788750] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.795120] ? trace_contention_end+0xb3/0x1a0
[ 1235.800906] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.806597] ? ww_mutex_lock+0x160/0x160
[ 1235.811797] msleep+0x1b/0x30
[ 1235.815922] napi_set_threaded+0x3a5/0x440
[ 1235.821320] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.826817] netif_set_threaded+0x257/0x400
[ 1235.832309] threaded_store+0x1b8/0x220
[ 1235.837413] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.842710] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.848000] ? sysfs_kf_write+0x65/0x160
[ 1235.853200] ? sysfs_file_ops+0x140/0x140
[ 1235.858496] kernfs_fop_write_iter+0x335/0x510
[ 1235.864283] ? kernfs_ops+0xf0/0xf0
[ 1235.868993] new_sync_write+0x333/0x750
[ 1235.874096] ? __lock_acquire+0x508/0xc10
[ 1235.879386] ? new_sync_read+0x740/0x740
[ 1235.884587] ? lock_acquire.part.0+0xbc/0x260
[ 1235.890274] ? ksys_write+0x116/0x250
[ 1235.895186] vfs_write+0x657/0x9b0
[ 1235.899802] ksys_write+0x116/0x250
[ 1235.904516] ? __ia32_sys_read+0xc0/0xc0
[ 1235.909716] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.915299] ? rcu_is_watching+0x15/0xd0
[ 1235.920497] do_syscall_64+0xf3/0x650
[ 1235.925406] ? trace_hardirqs_off+0xd/0x30
[ 1235.930801] ? exc_page_fault+0xda/0xf0
[ 1235.935903] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.942370] RIP: 0033:0x7f45e703964e
[ 1235.947179] RSP: 002b:00007ffea2670250 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.956474] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f45e703964e
[ 1235.965284] RDX: 0000000000000001 RSI: 000055ec8716ebf0 RDI: 0000000000000007
[ 1235.974093] RBP: 00007ffea2670260 R08: 0000000000000000 R09: 0000000000000000
[ 1235.982902] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f45e6eb2390
[ 1235.991709] R13: 000055ec8716ebf0 R14: 0000000000000007 R15: 00007f45e6e45120
[ 1236.000524]
[ 1236.003794] \x0aShowing all locks held in the system:
[ 1236.012334] 1 lock held by khungtaskd/184:
[ 1236.017729] #0: ffffffff934fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.029214] 4 locks held by python3/3432:
[ 1236.034509] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.044400] #1: ff1100013d066280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.055364] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.066521] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.076619] 4 locks held by python3/5422:
[ 1236.081917] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.091804] #1: ff110001865fac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.102764] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.113924] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.124015]
[ 1236.126482] =============================================\x0a
[ 1358.327368] INFO: task python3:5422 blocked for more than 491 seconds.
[ 1358.335508] Not tainted 7.1.0-rc7-ccpn-gb37f439f8a5a #1
[ 1358.342759] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.352351] task:python3 state:D stack:25552 pid:5422 tgid:5422 ppid:5421 task_flags:0x400100 flags:0x00080002
[ 1358.365558] Call Trace:
[ 1358.369098]
[ 1358.372251] __schedule+0xcc3/0x19c0
[ 1358.377069] ? io_schedule_timeout+0x140/0x140
[ 1358.382857] ? find_held_lock+0x2b/0x80
[ 1358.387962] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.393845] schedule+0xd8/0x260
[ 1358.398262] schedule_preempt_disabled+0x18/0x30
[ 1358.404243] __mutex_lock+0x6af/0x2010
[ 1358.409250] ? threaded_store+0x132/0x220
[ 1358.414551] ? ww_mutex_lock+0x160/0x160
[ 1358.419750] ? selinux_inode_free_security+0x330/0x330
[ 1358.426320] ? cap_capable+0x1d6/0x3e0
[ 1358.431330] ? threaded_store+0x132/0x220
[ 1358.436626] threaded_store+0x132/0x220
[ 1358.441730] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.447030] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.452333] ? sysfs_kf_write+0x65/0x160
[ 1358.457534] ? sysfs_file_ops+0x140/0x140
[ 1358.462830] kernfs_fop_write_iter+0x335/0x510
[ 1358.468616] ? kernfs_ops+0xf0/0xf0
[ 1358.473327] new_sync_write+0x333/0x750
[ 1358.478421] ? __lock_acquire+0x508/0xc10
[ 1358.483720] ? new_sync_read+0x740/0x740
[ 1358.488925] ? lock_acquire.part.0+0xbc/0x260
[ 1358.494611] ? ksys_write+0x116/0x250
[ 1358.499519] vfs_write+0x657/0x9b0
[ 1358.504134] ksys_write+0x116/0x250
[ 1358.508846] ? __ia32_sys_read+0xc0/0xc0
[ 1358.514043] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.519634] ? rcu_is_watching+0x15/0xd0
[ 1358.524838] do_syscall_64+0xf3/0x650
[ 1358.529743] ? trace_hardirqs_off+0xd/0x30
[ 1358.535138] ? exc_page_fault+0xda/0xf0
[ 1358.540240] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.546710] RIP: 0033:0x7f325ee1a64e
[ 1358.551520] RSP: 002b:00007ffc2a1d61b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.560808] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f325ee1a64e
[ 1358.569608] RDX: 0000000000000001 RSI: 0000561991474bf0 RDI: 0000000000000007
[ 1358.578406] RBP: 00007ffc2a1d61c0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.587207] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f325ec93390
[ 1358.596019] R13: 0000561991474bf0 R14: 0000000000000007 R15: 00007f325ec29120
[ 1358.604833]
[ 1358.608121] INFO: task python3:5422 is blocked on a mutex likely owned by task python3:3432.
[ 1358.618396] task:python3 state:D stack:25672 pid:3432 tgid:3432 ppid:3431 task_flags:0x400100 flags:0x00080802
[ 1358.631599] Call Trace:
[ 1358.635136]
[ 1358.638288] __schedule+0xcc3/0x19c0
[ 1358.643100] ? io_schedule_timeout+0x140/0x140
[ 1358.648885] ? find_held_lock+0x2b/0x80
[ 1358.653989] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.660166] schedule+0xd8/0x260
[ 1358.664585] schedule_timeout+0x13f/0x290
[ 1358.669881] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.676249] ? trace_contention_end+0xb3/0x1a0
[ 1358.682036] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.687728] ? ww_mutex_lock+0x160/0x160
[ 1358.692928] msleep+0x1b/0x30
[ 1358.697045] napi_set_threaded+0x3a5/0x440
[ 1358.702445] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.707937] netif_set_threaded+0x257/0x400
[ 1358.713423] threaded_store+0x1b8/0x220
[ 1358.718527] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.723824] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.729113] ? sysfs_kf_write+0x65/0x160
[ 1358.734313] ? sysfs_file_ops+0x140/0x140
[ 1358.739601] kernfs_fop_write_iter+0x335/0x510
[ 1358.745388] ? kernfs_ops+0xf0/0xf0
[ 1358.750099] new_sync_write+0x333/0x750
[ 1358.755204] ? __lock_acquire+0x508/0xc10
[ 1358.760499] ? new_sync_read+0x740/0x740
[ 1358.765702] ? lock_acquire.part.0+0xbc/0x260
[ 1358.771387] ? ksys_write+0x116/0x250
[ 1358.776296] vfs_write+0x657/0x9b0
[ 1358.780912] ksys_write+0x116/0x250
[ 1358.785626] ? __ia32_sys_read+0xc0/0xc0
[ 1358.790827] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.796414] ? rcu_is_watching+0x15/0xd0
[ 1358.801612] do_syscall_64+0xf3/0x650
[ 1358.806523] ? trace_hardirqs_off+0xd/0x30
[ 1358.811919] ? exc_page_fault+0xda/0xf0
[ 1358.817020] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.823489] RIP: 0033:0x7f45e703964e
[ 1358.828298] RSP: 002b:00007ffea2670250 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.837594] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f45e703964e
[ 1358.846403] RDX: 0000000000000001 RSI: 000055ec8716ebf0 RDI: 0000000000000007
[ 1358.855216] RBP: 00007ffea2670260 R08: 0000000000000000 R09: 0000000000000000
[ 1358.864024] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f45e6eb2390
[ 1358.872832] R13: 000055ec8716ebf0 R14: 0000000000000007 R15: 00007f45e6e45120
[ 1358.881645]
[ 1358.884918] \x0aShowing all locks held in the system:
[ 1358.893458] 1 lock held by khungtaskd/184:
[ 1358.898853] #0: ffffffff934fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.910333] 4 locks held by python3/3432:
[ 1358.915628] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.925518] #1: ff1100013d066280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.936483] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.947638] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.957731] 4 locks held by python3/5422:
[ 1358.963027] #0: ff1100012f32c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.972915] #1: ff110001865fac80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.983875] #2: ff11000121940b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.995033] #3: ff110001219a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.005129]
[ 1359.007595] =============================================\x0a