[ 781.416100] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 781.418431] nipa-hw-worker: [4/40] Retrying drivers/net:napi_threaded.py
[ 781.573929] kselftest: Running tests in drivers/net
[ 989.674660] INFO: task python3:5591 blocked for more than 122 seconds.
[ 989.682892] Not tainted 7.0.0-rc6-ekaz-g2729b1c38dea #1
[ 989.690142] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.699732] task:python3 state:D stack:25336 pid:5591 tgid:5591 ppid:5590 task_flags:0x400100 flags:0x00080000
[ 989.712942] Call Trace:
[ 989.712945]
[ 989.712948] __schedule+0xae3/0x14d0
[ 989.712960] ? io_schedule_timeout+0x140/0x140
[ 989.712963] ? find_held_lock+0x2b/0x80
[ 989.712970] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.741293] schedule+0xd4/0x260
[ 989.745710] schedule_preempt_disabled+0x18/0x30
[ 989.751689] __mutex_lock+0x6b2/0x1ee0
[ 989.756693] ? rcu_is_watching+0x15/0xd0
[ 989.761891] ? threaded_store+0x132/0x220
[ 989.767194] ? ww_mutex_lock+0x160/0x160
[ 989.772394] ? selinux_inode_free_security+0x330/0x330
[ 989.778964] ? cap_capable+0x1ed/0x3f0
[ 989.783977] ? threaded_store+0x132/0x220
[ 989.789277] threaded_store+0x132/0x220
[ 989.794381] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.799681] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.804978] ? sysfs_kf_write+0x65/0x160
[ 989.810179] ? sysfs_file_ops+0x140/0x140
[ 989.815476] kernfs_fop_write_iter+0x335/0x510
[ 989.821261] ? kernfs_ops+0xf0/0xf0
[ 989.825974] new_sync_write+0x333/0x750
[ 989.831079] ? __lock_acquire+0x508/0xc10
[ 989.836376] ? new_sync_read+0x740/0x740
[ 989.841569] ? lock_acquire.part.0+0xbc/0x260
[ 989.847256] ? ksys_write+0x116/0x250
[ 989.852156] vfs_write+0x657/0x9b0
[ 989.856769] ksys_write+0x116/0x250
[ 989.861479] ? __ia32_sys_read+0xc0/0xc0
[ 989.866681] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.872272] ? rcu_is_watching+0x15/0xd0
[ 989.877475] do_syscall_64+0xf3/0x5e0
[ 989.882381] ? trace_hardirqs_off+0xd/0x30
[ 989.887776] ? exc_page_fault+0xda/0xf0
[ 989.892878] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.899345] RIP: 0033:0x7f5c454f022e
[ 989.904154] RSP: 002b:00007fffa57b7940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.913455] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c454f022e
[ 989.922262] RDX: 0000000000000001 RSI: 000055a11ef63f40 RDI: 0000000000000007
[ 989.931070] RBP: 00007fffa57b7950 R08: 0000000000000000 R09: 0000000000000000
[ 989.939881] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c45368390
[ 989.948694] R13: 000055a11ef63f40 R14: 0000000000000007 R15: 0000000000000003
[ 989.957510]
[ 989.960805] INFO: task python3:5591 is blocked on a mutex likely owned by task python3:3556.
[ 989.971079] task:python3 state:D stack:25336 pid:3556 tgid:3556 ppid:3555 task_flags:0x400100 flags:0x00080802
[ 989.984283] Call Trace:
[ 989.987822]
[ 989.990973] __schedule+0xae3/0x14d0
[ 989.995784] ? io_schedule_timeout+0x140/0x140
[ 990.001569] ? find_held_lock+0x2b/0x80
[ 990.006664] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.012843] schedule+0xd4/0x260
[ 990.017261] schedule_timeout+0x13f/0x290
[ 990.022559] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 990.028920] ? trace_contention_end+0xb3/0x1a0
[ 990.034707] ? hrtimers_cpu_dying+0x260/0x260
[ 990.040400] ? ww_mutex_lock+0x160/0x160
[ 990.045600] msleep+0x1b/0x30
[ 990.049728] napi_set_threaded+0x3a5/0x440
[ 990.055126] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.060624] netif_set_threaded+0x257/0x400
[ 990.066119] threaded_store+0x1b8/0x220
[ 990.071220] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.076519] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.081815] ? sysfs_kf_write+0x65/0x160
[ 990.087014] ? sysfs_file_ops+0x140/0x140
[ 990.092302] kernfs_fop_write_iter+0x335/0x510
[ 990.098080] ? kernfs_ops+0xf0/0xf0
[ 990.102791] new_sync_write+0x333/0x750
[ 990.107894] ? __lock_acquire+0x508/0xc10
[ 990.113190] ? new_sync_read+0x740/0x740
[ 990.118391] ? lock_acquire.part.0+0xbc/0x260
[ 990.124078] ? ksys_write+0x116/0x250
[ 990.128987] vfs_write+0x657/0x9b0
[ 990.133600] ksys_write+0x116/0x250
[ 990.138311] ? __ia32_sys_read+0xc0/0xc0
[ 990.143509] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.149098] ? rcu_is_watching+0x15/0xd0
[ 990.154296] do_syscall_64+0xf3/0x5e0
[ 990.159205] ? trace_hardirqs_off+0xd/0x30
[ 990.164599] ? exc_page_fault+0xda/0xf0
[ 990.169703] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.176173] RIP: 0033:0x7fae2ed7722e
[ 990.180982] RSP: 002b:00007ffe02658460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.190271] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae2ed7722e
[ 990.199081] RDX: 0000000000000001 RSI: 0000555e781fff40 RDI: 0000000000000007
[ 990.207888] RBP: 00007ffe02658470 R08: 0000000000000000 R09: 0000000000000000
[ 990.216694] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae2ebef390
[ 990.225505] R13: 0000555e781fff40 R14: 0000000000000007 R15: 0000000000000003
[ 990.234320]
[ 990.237581] \x0aShowing all locks held in the system:
[ 990.246117] 1 lock held by khungtaskd/184:
[ 990.251510] #0: ffffffff9b8cdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.262982] 1 lock held by systemd-journal/802:
[ 990.268872] 4 locks held by python3/3556:
[ 990.274166] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.284055] #1: ff1100016c801a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.295017] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.306178] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.316285] 4 locks held by python3/5591:
[ 990.321593] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.331482] #1: ff110001dbd88888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.342449] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.353607] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.363694] 5 locks held by (systemd)/6282:
[ 990.369185] #0: ff1100012e4d43f8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.379073] #1: ff110001b4ac4a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.390036] #2: ffffffff9b937108 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 990.400706] #3: ffffffff9b768b50 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x6b0
[ 990.412342] #4: ffffffff9b936e10 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x6b0
[ 990.424773]
[ 990.427239] =============================================\x0a
[ 1112.554839] INFO: task python3:5591 blocked for more than 245 seconds.
[ 1112.562987] Not tainted 7.0.0-rc6-ekaz-g2729b1c38dea #1
[ 1112.570240] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.579828] task:python3 state:D stack:25336 pid:5591 tgid:5591 ppid:5590 task_flags:0x400100 flags:0x00080000
[ 1112.593025] Call Trace:
[ 1112.596563]
[ 1112.599714] __schedule+0xae3/0x14d0
[ 1112.604530] ? io_schedule_timeout+0x140/0x140
[ 1112.610314] ? find_held_lock+0x2b/0x80
[ 1112.615420] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.621297] schedule+0xd4/0x260
[ 1112.625717] schedule_preempt_disabled+0x18/0x30
[ 1112.631696] __mutex_lock+0x6b2/0x1ee0
[ 1112.636703] ? rcu_is_watching+0x15/0xd0
[ 1112.641902] ? threaded_store+0x132/0x220
[ 1112.647204] ? ww_mutex_lock+0x160/0x160
[ 1112.652405] ? selinux_inode_free_security+0x330/0x330
[ 1112.658977] ? cap_capable+0x1ed/0x3f0
[ 1112.663991] ? threaded_store+0x132/0x220
[ 1112.669289] threaded_store+0x132/0x220
[ 1112.674391] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.679683] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.684981] ? sysfs_kf_write+0x65/0x160
[ 1112.690182] ? sysfs_file_ops+0x140/0x140
[ 1112.695476] kernfs_fop_write_iter+0x335/0x510
[ 1112.701263] ? kernfs_ops+0xf0/0xf0
[ 1112.705975] new_sync_write+0x333/0x750
[ 1112.711077] ? __lock_acquire+0x508/0xc10
[ 1112.716374] ? new_sync_read+0x740/0x740
[ 1112.721581] ? lock_acquire.part.0+0xbc/0x260
[ 1112.727267] ? ksys_write+0x116/0x250
[ 1112.732176] vfs_write+0x657/0x9b0
[ 1112.736792] ksys_write+0x116/0x250
[ 1112.741503] ? __ia32_sys_read+0xc0/0xc0
[ 1112.746702] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.752291] ? rcu_is_watching+0x15/0xd0
[ 1112.757492] do_syscall_64+0xf3/0x5e0
[ 1112.762398] ? trace_hardirqs_off+0xd/0x30
[ 1112.767791] ? exc_page_fault+0xda/0xf0
[ 1112.772890] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.779359] RIP: 0033:0x7f5c454f022e
[ 1112.784167] RSP: 002b:00007fffa57b7940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.793464] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c454f022e
[ 1112.802264] RDX: 0000000000000001 RSI: 000055a11ef63f40 RDI: 0000000000000007
[ 1112.811062] RBP: 00007fffa57b7950 R08: 0000000000000000 R09: 0000000000000000
[ 1112.819873] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c45368390
[ 1112.828682] R13: 000055a11ef63f40 R14: 0000000000000007 R15: 0000000000000003
[ 1112.837496]
[ 1112.840787] INFO: task python3:5591 is blocked on a mutex likely owned by task python3:3556.
[ 1112.851062] task:python3 state:D stack:25336 pid:3556 tgid:3556 ppid:3555 task_flags:0x400100 flags:0x00080802
[ 1112.864264] Call Trace:
[ 1112.867805]
[ 1112.870956] __schedule+0xae3/0x14d0
[ 1112.875766] ? io_schedule_timeout+0x140/0x140
[ 1112.881549] ? find_held_lock+0x2b/0x80
[ 1112.886655] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.892835] schedule+0xd4/0x260
[ 1112.897255] schedule_timeout+0x13f/0x290
[ 1112.902553] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 1112.908912] ? trace_contention_end+0xb3/0x1a0
[ 1112.914698] ? hrtimers_cpu_dying+0x260/0x260
[ 1112.920387] ? ww_mutex_lock+0x160/0x160
[ 1112.925591] msleep+0x1b/0x30
[ 1112.929715] napi_set_threaded+0x3a5/0x440
[ 1112.935113] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.940597] netif_set_threaded+0x257/0x400
[ 1112.946092] threaded_store+0x1b8/0x220
[ 1112.951195] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.956491] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.961786] ? sysfs_kf_write+0x65/0x160
[ 1112.966986] ? sysfs_file_ops+0x140/0x140
[ 1112.972279] kernfs_fop_write_iter+0x335/0x510
[ 1112.978066] ? kernfs_ops+0xf0/0xf0
[ 1112.982779] new_sync_write+0x333/0x750
[ 1112.987882] ? __lock_acquire+0x508/0xc10
[ 1112.993180] ? new_sync_read+0x740/0x740
[ 1112.998383] ? lock_acquire.part.0+0xbc/0x260
[ 1113.004070] ? ksys_write+0x116/0x250
[ 1113.008982] vfs_write+0x657/0x9b0
[ 1113.013598] ksys_write+0x116/0x250
[ 1113.018311] ? __ia32_sys_read+0xc0/0xc0
[ 1113.023508] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.029099] ? rcu_is_watching+0x15/0xd0
[ 1113.034300] do_syscall_64+0xf3/0x5e0
[ 1113.039207] ? trace_hardirqs_off+0xd/0x30
[ 1113.044600] ? exc_page_fault+0xda/0xf0
[ 1113.049704] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.056171] RIP: 0033:0x7fae2ed7722e
[ 1113.060981] RSP: 002b:00007ffe02658460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.070277] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae2ed7722e
[ 1113.079086] RDX: 0000000000000001 RSI: 0000555e781fff40 RDI: 0000000000000007
[ 1113.087894] RBP: 00007ffe02658470 R08: 0000000000000000 R09: 0000000000000000
[ 1113.096695] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae2ebef390
[ 1113.105502] R13: 0000555e781fff40 R14: 0000000000000007 R15: 0000000000000003
[ 1113.114317]
[ 1113.117586] \x0aShowing all locks held in the system:
[ 1113.126121] 1 lock held by khungtaskd/184:
[ 1113.131515] #0: ffffffff9b8cdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.143002] 4 locks held by python3/3556:
[ 1113.148300] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.158193] #1: ff1100016c801a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.169153] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.180309] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.190404] 4 locks held by python3/5591:
[ 1113.195701] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.205593] #1: ff110001dbd88888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.216554] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.227714] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.237807]
[ 1113.240287] =============================================\x0a
[ 1235.435844] INFO: task python3:5591 blocked for more than 368 seconds.
[ 1235.443995] Not tainted 7.0.0-rc6-ekaz-g2729b1c38dea #1
[ 1235.451246] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.460837] task:python3 state:D stack:25336 pid:5591 tgid:5591 ppid:5590 task_flags:0x400100 flags:0x00080002
[ 1235.474041] Call Trace:
[ 1235.477580]
[ 1235.480732] __schedule+0xae3/0x14d0
[ 1235.485549] ? io_schedule_timeout+0x140/0x140
[ 1235.491334] ? find_held_lock+0x2b/0x80
[ 1235.496441] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.502325] schedule+0xd4/0x260
[ 1235.506747] schedule_preempt_disabled+0x18/0x30
[ 1235.512727] __mutex_lock+0x6b2/0x1ee0
[ 1235.517726] ? rcu_is_watching+0x15/0xd0
[ 1235.522926] ? threaded_store+0x132/0x220
[ 1235.528229] ? ww_mutex_lock+0x160/0x160
[ 1235.533427] ? selinux_inode_free_security+0x330/0x330
[ 1235.539992] ? cap_capable+0x1ed/0x3f0
[ 1235.545000] ? threaded_store+0x132/0x220
[ 1235.550301] threaded_store+0x132/0x220
[ 1235.555403] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.560701] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.566000] ? sysfs_kf_write+0x65/0x160
[ 1235.571199] ? sysfs_file_ops+0x140/0x140
[ 1235.576495] kernfs_fop_write_iter+0x335/0x510
[ 1235.582281] ? kernfs_ops+0xf0/0xf0
[ 1235.587058] new_sync_write+0x333/0x750
[ 1235.592164] ? __lock_acquire+0x508/0xc10
[ 1235.597460] ? new_sync_read+0x740/0x740
[ 1235.602734] ? lock_acquire.part.0+0xbc/0x260
[ 1235.608422] ? ksys_write+0x116/0x250
[ 1235.613335] vfs_write+0x657/0x9b0
[ 1235.617951] ksys_write+0x116/0x250
[ 1235.622666] ? __ia32_sys_read+0xc0/0xc0
[ 1235.627863] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.633455] ? rcu_is_watching+0x15/0xd0
[ 1235.638658] do_syscall_64+0xf3/0x5e0
[ 1235.643567] ? trace_hardirqs_off+0xd/0x30
[ 1235.648962] ? exc_page_fault+0xda/0xf0
[ 1235.654065] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.660534] RIP: 0033:0x7f5c454f022e
[ 1235.665418] RSP: 002b:00007fffa57b7940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.674717] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c454f022e
[ 1235.683528] RDX: 0000000000000001 RSI: 000055a11ef63f40 RDI: 0000000000000007
[ 1235.692338] RBP: 00007fffa57b7950 R08: 0000000000000000 R09: 0000000000000000
[ 1235.701147] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c45368390
[ 1235.709955] R13: 000055a11ef63f40 R14: 0000000000000007 R15: 0000000000000003
[ 1235.718758]
[ 1235.722043] INFO: task python3:5591 is blocked on a mutex likely owned by task python3:3556.
[ 1235.732317] task:python3 state:D stack:25336 pid:3556 tgid:3556 ppid:3555 task_flags:0x400100 flags:0x00080802
[ 1235.745518] Call Trace:
[ 1235.749058]
[ 1235.752212] __schedule+0xae3/0x14d0
[ 1235.757023] ? io_schedule_timeout+0x140/0x140
[ 1235.762808] ? find_held_lock+0x2b/0x80
[ 1235.767904] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.774083] schedule+0xd4/0x260
[ 1235.778501] schedule_timeout+0x13f/0x290
[ 1235.783797] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 1235.790167] ? trace_contention_end+0xb3/0x1a0
[ 1235.795945] ? hrtimers_cpu_dying+0x260/0x260
[ 1235.801638] ? ww_mutex_lock+0x160/0x160
[ 1235.806837] msleep+0x1b/0x30
[ 1235.810961] napi_set_threaded+0x3a5/0x440
[ 1235.816423] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.821921] netif_set_threaded+0x257/0x400
[ 1235.827414] threaded_store+0x1b8/0x220
[ 1235.832520] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.837820] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.843117] ? sysfs_kf_write+0x65/0x160
[ 1235.848316] ? sysfs_file_ops+0x140/0x140
[ 1235.853609] kernfs_fop_write_iter+0x335/0x510
[ 1235.859397] ? kernfs_ops+0xf0/0xf0
[ 1235.864108] new_sync_write+0x333/0x750
[ 1235.869209] ? __lock_acquire+0x508/0xc10
[ 1235.874504] ? new_sync_read+0x740/0x740
[ 1235.879707] ? lock_acquire.part.0+0xbc/0x260
[ 1235.885395] ? ksys_write+0x116/0x250
[ 1235.890307] vfs_write+0x657/0x9b0
[ 1235.894926] ksys_write+0x116/0x250
[ 1235.899640] ? __ia32_sys_read+0xc0/0xc0
[ 1235.904838] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.910429] ? rcu_is_watching+0x15/0xd0
[ 1235.915629] do_syscall_64+0xf3/0x5e0
[ 1235.920535] ? trace_hardirqs_off+0xd/0x30
[ 1235.925927] ? exc_page_fault+0xda/0xf0
[ 1235.931030] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.937496] RIP: 0033:0x7fae2ed7722e
[ 1235.942305] RSP: 002b:00007ffe02658460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.951604] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae2ed7722e
[ 1235.960416] RDX: 0000000000000001 RSI: 0000555e781fff40 RDI: 0000000000000007
[ 1235.969224] RBP: 00007ffe02658470 R08: 0000000000000000 R09: 0000000000000000
[ 1235.978024] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae2ebef390
[ 1235.986830] R13: 0000555e781fff40 R14: 0000000000000007 R15: 0000000000000003
[ 1235.995645]
[ 1235.998914] \x0aShowing all locks held in the system:
[ 1236.007455] 1 lock held by khungtaskd/184:
[ 1236.012850] #0: ffffffff9b8cdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.024335] 4 locks held by python3/3556:
[ 1236.029633] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.039525] #1: ff1100016c801a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.050488] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.061645] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.071732] 4 locks held by python3/5591:
[ 1236.077030] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.086919] #1: ff110001dbd88888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.097884] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.109041] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.119126]
[ 1236.121592] =============================================\x0a
[ 1358.317946] INFO: task python3:5591 blocked for more than 491 seconds.
[ 1358.326093] Not tainted 7.0.0-rc6-ekaz-g2729b1c38dea #1
[ 1358.333344] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.342935] task:python3 state:D stack:25336 pid:5591 tgid:5591 ppid:5590 task_flags:0x400100 flags:0x00080002
[ 1358.356142] Call Trace:
[ 1358.359683]
[ 1358.362834] __schedule+0xae3/0x14d0
[ 1358.367653] ? io_schedule_timeout+0x140/0x140
[ 1358.373439] ? find_held_lock+0x2b/0x80
[ 1358.378545] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.384428] schedule+0xd4/0x260
[ 1358.388848] schedule_preempt_disabled+0x18/0x30
[ 1358.394820] __mutex_lock+0x6b2/0x1ee0
[ 1358.399827] ? rcu_is_watching+0x15/0xd0
[ 1358.405028] ? threaded_store+0x132/0x220
[ 1358.410331] ? ww_mutex_lock+0x160/0x160
[ 1358.415530] ? selinux_inode_free_security+0x330/0x330
[ 1358.422103] ? cap_capable+0x1ed/0x3f0
[ 1358.427111] ? threaded_store+0x132/0x220
[ 1358.432407] threaded_store+0x132/0x220
[ 1358.437510] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.442811] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.448112] ? sysfs_kf_write+0x65/0x160
[ 1358.453313] ? sysfs_file_ops+0x140/0x140
[ 1358.458608] kernfs_fop_write_iter+0x335/0x510
[ 1358.464396] ? kernfs_ops+0xf0/0xf0
[ 1358.469106] new_sync_write+0x333/0x750
[ 1358.474209] ? __lock_acquire+0x508/0xc10
[ 1358.479505] ? new_sync_read+0x740/0x740
[ 1358.484705] ? lock_acquire.part.0+0xbc/0x260
[ 1358.490381] ? ksys_write+0x116/0x250
[ 1358.495294] vfs_write+0x657/0x9b0
[ 1358.499910] ksys_write+0x116/0x250
[ 1358.504623] ? __ia32_sys_read+0xc0/0xc0
[ 1358.509823] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.515406] ? rcu_is_watching+0x15/0xd0
[ 1358.520608] do_syscall_64+0xf3/0x5e0
[ 1358.525516] ? trace_hardirqs_off+0xd/0x30
[ 1358.530911] ? exc_page_fault+0xda/0xf0
[ 1358.536013] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.542482] RIP: 0033:0x7f5c454f022e
[ 1358.547294] RSP: 002b:00007fffa57b7940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.556593] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c454f022e
[ 1358.565403] RDX: 0000000000000001 RSI: 000055a11ef63f40 RDI: 0000000000000007
[ 1358.574213] RBP: 00007fffa57b7950 R08: 0000000000000000 R09: 0000000000000000
[ 1358.583023] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c45368390
[ 1358.591831] R13: 000055a11ef63f40 R14: 0000000000000007 R15: 0000000000000003
[ 1358.600646]
[ 1358.603918] INFO: task python3:5591 is blocked on a mutex likely owned by task python3:3556.
[ 1358.614191] task:python3 state:D stack:25336 pid:3556 tgid:3556 ppid:3555 task_flags:0x400100 flags:0x00080802
[ 1358.627394] Call Trace:
[ 1358.630933]
[ 1358.634083] __schedule+0xae3/0x14d0
[ 1358.638894] ? io_schedule_timeout+0x140/0x140
[ 1358.644679] ? find_held_lock+0x2b/0x80
[ 1358.649773] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.655955] schedule+0xd4/0x260
[ 1358.660374] schedule_timeout+0x13f/0x290
[ 1358.665662] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 1358.672031] ? trace_contention_end+0xb3/0x1a0
[ 1358.677818] ? hrtimers_cpu_dying+0x260/0x260
[ 1358.683510] ? ww_mutex_lock+0x160/0x160
[ 1358.688712] msleep+0x1b/0x30
[ 1358.692838] napi_set_threaded+0x3a5/0x440
[ 1358.698236] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.703733] netif_set_threaded+0x257/0x400
[ 1358.709226] threaded_store+0x1b8/0x220
[ 1358.714320] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.719619] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.724907] ? sysfs_kf_write+0x65/0x160
[ 1358.730106] ? sysfs_file_ops+0x140/0x140
[ 1358.735402] kernfs_fop_write_iter+0x335/0x510
[ 1358.741189] ? kernfs_ops+0xf0/0xf0
[ 1358.745900] new_sync_write+0x333/0x750
[ 1358.751003] ? __lock_acquire+0x508/0xc10
[ 1358.756299] ? new_sync_read+0x740/0x740
[ 1358.761500] ? lock_acquire.part.0+0xbc/0x260
[ 1358.767186] ? ksys_write+0x116/0x250
[ 1358.772099] vfs_write+0x657/0x9b0
[ 1358.776714] ksys_write+0x116/0x250
[ 1358.781417] ? __ia32_sys_read+0xc0/0xc0
[ 1358.786614] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.792206] ? rcu_is_watching+0x15/0xd0
[ 1358.797407] do_syscall_64+0xf3/0x5e0
[ 1358.802312] ? trace_hardirqs_off+0xd/0x30
[ 1358.807706] ? exc_page_fault+0xda/0xf0
[ 1358.812806] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.819274] RIP: 0033:0x7fae2ed7722e
[ 1358.824082] RSP: 002b:00007ffe02658460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.833382] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae2ed7722e
[ 1358.842192] RDX: 0000000000000001 RSI: 0000555e781fff40 RDI: 0000000000000007
[ 1358.850999] RBP: 00007ffe02658470 R08: 0000000000000000 R09: 0000000000000000
[ 1358.859809] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae2ebef390
[ 1358.868617] R13: 0000555e781fff40 R14: 0000000000000007 R15: 0000000000000003
[ 1358.877431]
[ 1358.880701] \x0aShowing all locks held in the system:
[ 1358.889237] 1 lock held by khungtaskd/184:
[ 1358.894632] #0: ffffffff9b8cdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.906114] 4 locks held by python3/3556:
[ 1358.911409] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.921297] #1: ff1100016c801a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.932262] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.943418] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.953513] 4 locks held by python3/5591:
[ 1358.958810] #0: ff1100012e3943f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.968699] #1: ff110001dbd88888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.979661] #2: ff110001213790f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.990821] #3: ff110001213e1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.000917]
[ 1359.003385] =============================================\x0a