[ 761.390001] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 761.392531] nipa-hw-worker: [5/40] Retrying drivers/net:napi_threaded.py
[ 761.551350] kselftest: Running tests in drivers/net
[ 989.755981] INFO: task python3:5229 blocked for more than 122 seconds.
[ 989.764213] Not tainted 7.0.0-rc5-ctje-gdce8fbbe00f0 #1
[ 989.771462] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.781043] task:python3 state:D stack:25312 pid:5229 tgid:5229 ppid:5228 task_flags:0x400100 flags:0x00080000
[ 989.794248] Call Trace:
[ 989.797788]
[ 989.800939] __schedule+0xad9/0x1490
[ 989.805756] ? io_schedule_timeout+0x130/0x130
[ 989.811544] ? find_held_lock+0x2b/0x80
[ 989.816651] ? __lock_release.isra.0+0x59/0x170
[ 989.822538] schedule+0xd4/0x260
[ 989.826956] schedule_preempt_disabled+0x18/0x30
[ 989.832937] __mutex_lock+0xd76/0x1f20
[ 989.837945] ? threaded_store+0x132/0x220
[ 989.843246] ? ww_mutex_lock+0x160/0x160
[ 989.848446] ? selinux_inode_free_security+0x310/0x310
[ 989.855017] ? cap_capable+0x1ed/0x3f0
[ 989.860025] ? sysfs_file_ops+0x140/0x140
[ 989.865323] ? threaded_store+0x132/0x220
[ 989.870618] threaded_store+0x132/0x220
[ 989.875724] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.881218] ? sysfs_file_ops+0x140/0x140
[ 989.886517] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.891814] ? sysfs_kf_write+0x65/0x160
[ 989.897005] kernfs_fop_write_iter+0x335/0x510
[ 989.902781] ? kernfs_ops+0xf0/0xf0
[ 989.907494] new_sync_write+0x308/0x6f0
[ 989.912597] ? new_sync_read+0x6f0/0x6f0
[ 989.917788] ? lock_acquire.part.0+0xbc/0x260
[ 989.923476] ? ksys_write+0xff/0x200
[ 989.928289] vfs_write+0x62a/0x980
[ 989.932905] ksys_write+0xff/0x200
[ 989.937522] ? __ia32_sys_read+0xc0/0xc0
[ 989.942719] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.948308] ? rcu_is_watching+0x15/0xd0
[ 989.953510] do_syscall_64+0xf3/0x5e0
[ 989.958418] ? exc_page_fault+0x9e/0xb0
[ 989.963518] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.969987] RIP: 0033:0x7f07e73cdc5e
[ 989.974796] RSP: 002b:00007fff8e3b4430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.984104] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07e73cdc5e
[ 989.992913] RDX: 0000000000000001 RSI: 000055d0e1009760 RDI: 0000000000000007
[ 990.001721] RBP: 00007fff8e3b4440 R08: 0000000000000000 R09: 0000000000000000
[ 990.010531] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07e7268e80
[ 990.019339] R13: 000055d0e1009760 R14: 0000000000000007 R15: 00007f07e7aceb98
[ 990.028144]
[ 990.031411] INFO: task python3:5229 is blocked on a mutex likely owned by task python3:3254.
[ 990.041686] task:python3 state:D stack:25312 pid:3254 tgid:3254 ppid:3253 task_flags:0x400100 flags:0x00080802
[ 990.054890] Call Trace:
[ 990.058427]
[ 990.061577] __schedule+0xad9/0x1490
[ 990.066387] ? io_schedule_timeout+0x130/0x130
[ 990.072165] ? find_held_lock+0x2b/0x80
[ 990.077267] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.083449] schedule+0xd4/0x260
[ 990.087867] schedule_timeout+0x103/0x260
[ 990.093166] ? rcu_is_watching+0x15/0xd0
[ 990.098366] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.104736] ? hrtimers_cpu_dying+0x250/0x250
[ 990.110416] ? ww_mutex_lock+0x160/0x160
[ 990.115617] msleep+0x1b/0x30
[ 990.119743] napi_set_threaded+0x376/0x410
[ 990.125137] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.130634] netif_set_threaded+0x254/0x400
[ 990.136127] ? sysfs_file_ops+0x140/0x140
[ 990.141423] threaded_store+0x1bb/0x220
[ 990.146524] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.152017] ? sysfs_file_ops+0x140/0x140
[ 990.157313] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.162608] ? sysfs_kf_write+0x65/0x160
[ 990.167814] kernfs_fop_write_iter+0x335/0x510
[ 990.173598] ? kernfs_ops+0xf0/0xf0
[ 990.178310] new_sync_write+0x308/0x6f0
[ 990.183414] ? new_sync_read+0x6f0/0x6f0
[ 990.188614] ? lock_acquire.part.0+0xbc/0x260
[ 990.194292] ? ksys_write+0xff/0x200
[ 990.199104] vfs_write+0x62a/0x980
[ 990.203719] ksys_write+0xff/0x200
[ 990.208332] ? __ia32_sys_read+0xc0/0xc0
[ 990.213532] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.219122] ? rcu_is_watching+0x15/0xd0
[ 990.224313] do_syscall_64+0xf3/0x5e0
[ 990.229220] ? exc_page_fault+0x9e/0xb0
[ 990.234320] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.240786] RIP: 0033:0x7f8663516c5e
[ 990.245594] RSP: 002b:00007ffce7020950 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.254889] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8663516c5e
[ 990.263698] RDX: 0000000000000001 RSI: 000055b4d6854760 RDI: 0000000000000007
[ 990.272504] RBP: 00007ffce7020960 R08: 0000000000000000 R09: 0000000000000000
[ 990.281311] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f86633b1e80
[ 990.290118] R13: 000055b4d6854760 R14: 0000000000000007 R15: 00007f8663342480
[ 990.298922]
[ 990.302184] \x0aShowing all locks held in the system:
[ 990.310720] 1 lock held by khungtaskd/184:
[ 990.316114] #0: ffffffffadcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.327589] 3 locks held by systemd-journal/805:
[ 990.333570] #0: ff1100180e4372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 990.343460] #1: ffffffffadec91e0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.356276] #2: ff110001a3a83360 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 990.367739] 4 locks held by python3/3254:
[ 990.373034] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.382827] #1: ff1100013df1a088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.393789] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.404950] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.415059] 4 locks held by python3/5229:
[ 990.420359] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.430144] #1: ff11000122a45688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.441108] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.452268] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.462359]
[ 990.464827] =============================================\x0a
[ 1112.638294] INFO: task python3:5229 blocked for more than 245 seconds.
[ 1112.646444] Not tainted 7.0.0-rc5-ctje-gdce8fbbe00f0 #1
[ 1112.653692] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.663271] task:python3 state:D stack:25312 pid:5229 tgid:5229 ppid:5228 task_flags:0x400100 flags:0x00080000
[ 1112.676474] Call Trace:
[ 1112.680011]
[ 1112.683161] __schedule+0xad9/0x1490
[ 1112.687976] ? io_schedule_timeout+0x130/0x130
[ 1112.693763] ? find_held_lock+0x2b/0x80
[ 1112.698869] ? __lock_release.isra.0+0x59/0x170
[ 1112.704757] schedule+0xd4/0x260
[ 1112.709175] schedule_preempt_disabled+0x18/0x30
[ 1112.715152] __mutex_lock+0xd76/0x1f20
[ 1112.720159] ? threaded_store+0x132/0x220
[ 1112.725456] ? ww_mutex_lock+0x160/0x160
[ 1112.730652] ? selinux_inode_free_security+0x310/0x310
[ 1112.737221] ? cap_capable+0x1ed/0x3f0
[ 1112.742227] ? sysfs_file_ops+0x140/0x140
[ 1112.747528] ? threaded_store+0x132/0x220
[ 1112.752892] threaded_store+0x132/0x220
[ 1112.757993] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.763486] ? sysfs_file_ops+0x140/0x140
[ 1112.768782] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.774079] ? sysfs_kf_write+0x65/0x160
[ 1112.779279] kernfs_fop_write_iter+0x335/0x510
[ 1112.785055] ? kernfs_ops+0xf0/0xf0
[ 1112.789763] new_sync_write+0x308/0x6f0
[ 1112.794867] ? new_sync_read+0x6f0/0x6f0
[ 1112.800058] ? lock_acquire.part.0+0xbc/0x260
[ 1112.805742] ? ksys_write+0xff/0x200
[ 1112.810551] vfs_write+0x62a/0x980
[ 1112.815164] ksys_write+0xff/0x200
[ 1112.819779] ? __ia32_sys_read+0xc0/0xc0
[ 1112.825043] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.830633] ? rcu_is_watching+0x15/0xd0
[ 1112.835836] do_syscall_64+0xf3/0x5e0
[ 1112.840741] ? exc_page_fault+0x9e/0xb0
[ 1112.845842] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.852300] RIP: 0033:0x7f07e73cdc5e
[ 1112.857107] RSP: 002b:00007fff8e3b4430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.866402] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07e73cdc5e
[ 1112.875207] RDX: 0000000000000001 RSI: 000055d0e1009760 RDI: 0000000000000007
[ 1112.884014] RBP: 00007fff8e3b4440 R08: 0000000000000000 R09: 0000000000000000
[ 1112.892821] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07e7268e80
[ 1112.901628] R13: 000055d0e1009760 R14: 0000000000000007 R15: 00007f07e7aceb98
[ 1112.910441]
[ 1112.913727] INFO: task python3:5229 is blocked on a mutex likely owned by task python3:3254.
[ 1112.924001] task:python3 state:D stack:25312 pid:3254 tgid:3254 ppid:3253 task_flags:0x400100 flags:0x00080802
[ 1112.937203] Call Trace:
[ 1112.940739]
[ 1112.943887] __schedule+0xad9/0x1490
[ 1112.948699] ? io_schedule_timeout+0x130/0x130
[ 1112.954480] ? find_held_lock+0x2b/0x80
[ 1112.959581] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.965753] schedule+0xd4/0x260
[ 1112.970168] schedule_timeout+0x103/0x260
[ 1112.975463] ? rcu_is_watching+0x15/0xd0
[ 1112.980659] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.987027] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.992715] ? ww_mutex_lock+0x160/0x160
[ 1112.997913] msleep+0x1b/0x30
[ 1113.002107] napi_set_threaded+0x376/0x410
[ 1113.007503] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.012994] netif_set_threaded+0x254/0x400
[ 1113.018487] ? sysfs_file_ops+0x140/0x140
[ 1113.023783] threaded_store+0x1bb/0x220
[ 1113.028886] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.034368] ? sysfs_file_ops+0x140/0x140
[ 1113.039663] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.044958] ? sysfs_kf_write+0x65/0x160
[ 1113.050158] kernfs_fop_write_iter+0x335/0x510
[ 1113.055943] ? kernfs_ops+0xf0/0xf0
[ 1113.060645] new_sync_write+0x308/0x6f0
[ 1113.065747] ? new_sync_read+0x6f0/0x6f0
[ 1113.070937] ? lock_acquire.part.0+0xbc/0x260
[ 1113.076620] ? ksys_write+0xff/0x200
[ 1113.081433] vfs_write+0x62a/0x980
[ 1113.086048] ksys_write+0xff/0x200
[ 1113.090652] ? __ia32_sys_read+0xc0/0xc0
[ 1113.095849] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.101438] ? rcu_is_watching+0x15/0xd0
[ 1113.106705] do_syscall_64+0xf3/0x5e0
[ 1113.111610] ? exc_page_fault+0x9e/0xb0
[ 1113.116710] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.123177] RIP: 0033:0x7f8663516c5e
[ 1113.127984] RSP: 002b:00007ffce7020950 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.137279] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8663516c5e
[ 1113.146087] RDX: 0000000000000001 RSI: 000055b4d6854760 RDI: 0000000000000007
[ 1113.154886] RBP: 00007ffce7020960 R08: 0000000000000000 R09: 0000000000000000
[ 1113.163696] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f86633b1e80
[ 1113.172502] R13: 000055b4d6854760 R14: 0000000000000007 R15: 00007f8663342480
[ 1113.181317]
[ 1113.184582] \x0aShowing all locks held in the system:
[ 1113.193119] 1 lock held by khungtaskd/184:
[ 1113.198512] #0: ffffffffadcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.209994] 4 locks held by python3/3254:
[ 1113.215289] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.225080] #1: ff1100013df1a088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.236040] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.247197] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.257296] 4 locks held by python3/5229:
[ 1113.262591] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.272379] #1: ff11000122a45688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.283342] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.294499] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.304590] 1 lock held by systemd/6310:
[ 1113.309781]
[ 1113.312244] =============================================\x0a
[ 1235.518585] INFO: task python3:5229 blocked for more than 368 seconds.
[ 1235.526733] Not tainted 7.0.0-rc5-ctje-gdce8fbbe00f0 #1
[ 1235.533983] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.543568] task:python3 state:D stack:25312 pid:5229 tgid:5229 ppid:5228 task_flags:0x400100 flags:0x00080002
[ 1235.556771] Call Trace:
[ 1235.560308]
[ 1235.563460] __schedule+0xad9/0x1490
[ 1235.568273] ? io_schedule_timeout+0x130/0x130
[ 1235.574058] ? find_held_lock+0x2b/0x80
[ 1235.579165] ? __lock_release.isra.0+0x59/0x170
[ 1235.585047] schedule+0xd4/0x260
[ 1235.589463] schedule_preempt_disabled+0x18/0x30
[ 1235.595439] __mutex_lock+0xd76/0x1f20
[ 1235.600442] ? threaded_store+0x132/0x220
[ 1235.605741] ? ww_mutex_lock+0x160/0x160
[ 1235.610939] ? selinux_inode_free_security+0x310/0x310
[ 1235.617500] ? cap_capable+0x1ed/0x3f0
[ 1235.622506] ? sysfs_file_ops+0x140/0x140
[ 1235.627808] ? threaded_store+0x132/0x220
[ 1235.633104] threaded_store+0x132/0x220
[ 1235.638206] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.643696] ? sysfs_file_ops+0x140/0x140
[ 1235.648992] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.654286] ? sysfs_kf_write+0x65/0x160
[ 1235.659485] kernfs_fop_write_iter+0x335/0x510
[ 1235.665271] ? kernfs_ops+0xf0/0xf0
[ 1235.669980] new_sync_write+0x308/0x6f0
[ 1235.675085] ? new_sync_read+0x6f0/0x6f0
[ 1235.680285] ? lock_acquire.part.0+0xbc/0x260
[ 1235.685971] ? ksys_write+0xff/0x200
[ 1235.690780] vfs_write+0x62a/0x980
[ 1235.695394] ksys_write+0xff/0x200
[ 1235.700006] ? __ia32_sys_read+0xc0/0xc0
[ 1235.705201] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.710792] ? rcu_is_watching+0x15/0xd0
[ 1235.715992] do_syscall_64+0xf3/0x5e0
[ 1235.720900] ? exc_page_fault+0x9e/0xb0
[ 1235.726001] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.732462] RIP: 0033:0x7f07e73cdc5e
[ 1235.737272] RSP: 002b:00007fff8e3b4430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.746561] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07e73cdc5e
[ 1235.755368] RDX: 0000000000000001 RSI: 000055d0e1009760 RDI: 0000000000000007
[ 1235.764177] RBP: 00007fff8e3b4440 R08: 0000000000000000 R09: 0000000000000000
[ 1235.772981] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07e7268e80
[ 1235.781787] R13: 000055d0e1009760 R14: 0000000000000007 R15: 00007f07e7aceb98
[ 1235.790600]
[ 1235.793871] INFO: task python3:5229 is blocked on a mutex likely owned by task python3:3254.
[ 1235.804143] task:python3 state:D stack:25312 pid:3254 tgid:3254 ppid:3253 task_flags:0x400100 flags:0x00080802
[ 1235.817344] Call Trace:
[ 1235.820880]
[ 1235.824029] __schedule+0xad9/0x1490
[ 1235.828839] ? io_schedule_timeout+0x130/0x130
[ 1235.834620] ? find_held_lock+0x2b/0x80
[ 1235.839720] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.845900] schedule+0xd4/0x260
[ 1235.850318] schedule_timeout+0x103/0x260
[ 1235.855614] ? rcu_is_watching+0x15/0xd0
[ 1235.860810] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.867178] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.872866] ? ww_mutex_lock+0x160/0x160
[ 1235.878054] msleep+0x1b/0x30
[ 1235.882182] napi_set_threaded+0x376/0x410
[ 1235.887567] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.893061] netif_set_threaded+0x254/0x400
[ 1235.898555] ? sysfs_file_ops+0x140/0x140
[ 1235.903850] threaded_store+0x1bb/0x220
[ 1235.908953] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.914443] ? sysfs_file_ops+0x140/0x140
[ 1235.919737] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.925033] ? sysfs_kf_write+0x65/0x160
[ 1235.930230] kernfs_fop_write_iter+0x335/0x510
[ 1235.936012] ? kernfs_ops+0xf0/0xf0
[ 1235.940721] new_sync_write+0x308/0x6f0
[ 1235.945822] ? new_sync_read+0x6f0/0x6f0
[ 1235.951020] ? lock_acquire.part.0+0xbc/0x260
[ 1235.956705] ? ksys_write+0xff/0x200
[ 1235.961516] vfs_write+0x62a/0x980
[ 1235.966131] ksys_write+0xff/0x200
[ 1235.970744] ? __ia32_sys_read+0xc0/0xc0
[ 1235.975942] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.981530] ? rcu_is_watching+0x15/0xd0
[ 1235.986731] do_syscall_64+0xf3/0x5e0
[ 1235.991636] ? exc_page_fault+0x9e/0xb0
[ 1235.996736] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1236.003202] RIP: 0033:0x7f8663516c5e
[ 1236.008007] RSP: 002b:00007ffce7020950 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.017300] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8663516c5e
[ 1236.026105] RDX: 0000000000000001 RSI: 000055b4d6854760 RDI: 0000000000000007
[ 1236.034912] RBP: 00007ffce7020960 R08: 0000000000000000 R09: 0000000000000000
[ 1236.043718] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f86633b1e80
[ 1236.052525] R13: 000055b4d6854760 R14: 0000000000000007 R15: 00007f8663342480
[ 1236.061337]
[ 1236.064602] \x0aShowing all locks held in the system:
[ 1236.073139] 1 lock held by khungtaskd/184:
[ 1236.078532] #0: ffffffffadcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.090013] 4 locks held by python3/3254:
[ 1236.095309] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.105100] #1: ff1100013df1a088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.116057] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.127214] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.137297] 4 locks held by python3/5229:
[ 1236.142593] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.152381] #1: ff11000122a45688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.163340] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.174500] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.184589]
[ 1236.187055] =============================================\x0a
[ 1358.399877] INFO: task python3:5229 blocked for more than 491 seconds.
[ 1358.408025] Not tainted 7.0.0-rc5-ctje-gdce8fbbe00f0 #1
[ 1358.415277] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.424865] task:python3 state:D stack:25312 pid:5229 tgid:5229 ppid:5228 task_flags:0x400100 flags:0x00080002
[ 1358.438073] Call Trace:
[ 1358.441612]
[ 1358.444763] __schedule+0xad9/0x1490
[ 1358.449580] ? io_schedule_timeout+0x130/0x130
[ 1358.455365] ? find_held_lock+0x2b/0x80
[ 1358.460474] ? __lock_release.isra.0+0x59/0x170
[ 1358.466362] schedule+0xd4/0x260
[ 1358.470781] schedule_preempt_disabled+0x18/0x30
[ 1358.476763] __mutex_lock+0xd76/0x1f20
[ 1358.481768] ? threaded_store+0x132/0x220
[ 1358.487067] ? ww_mutex_lock+0x160/0x160
[ 1358.492265] ? selinux_inode_free_security+0x310/0x310
[ 1358.498825] ? cap_capable+0x1ed/0x3f0
[ 1358.503833] ? sysfs_file_ops+0x140/0x140
[ 1358.509133] ? threaded_store+0x132/0x220
[ 1358.514432] threaded_store+0x132/0x220
[ 1358.519536] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.525033] ? sysfs_file_ops+0x140/0x140
[ 1358.530331] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.535630] ? sysfs_kf_write+0x65/0x160
[ 1358.540829] kernfs_fop_write_iter+0x335/0x510
[ 1358.546615] ? kernfs_ops+0xf0/0xf0
[ 1358.551328] new_sync_write+0x308/0x6f0
[ 1358.556434] ? new_sync_read+0x6f0/0x6f0
[ 1358.561634] ? lock_acquire.part.0+0xbc/0x260
[ 1358.567310] ? ksys_write+0xff/0x200
[ 1358.572125] vfs_write+0x62a/0x980
[ 1358.576739] ksys_write+0xff/0x200
[ 1358.581353] ? __ia32_sys_read+0xc0/0xc0
[ 1358.586551] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.592143] ? rcu_is_watching+0x15/0xd0
[ 1358.597348] do_syscall_64+0xf3/0x5e0
[ 1358.602253] ? exc_page_fault+0x9e/0xb0
[ 1358.607353] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.613825] RIP: 0033:0x7f07e73cdc5e
[ 1358.618637] RSP: 002b:00007fff8e3b4430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.627935] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f07e73cdc5e
[ 1358.636745] RDX: 0000000000000001 RSI: 000055d0e1009760 RDI: 0000000000000007
[ 1358.645554] RBP: 00007fff8e3b4440 R08: 0000000000000000 R09: 0000000000000000
[ 1358.654363] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f07e7268e80
[ 1358.663173] R13: 000055d0e1009760 R14: 0000000000000007 R15: 00007f07e7aceb98
[ 1358.671989]
[ 1358.675269] INFO: task python3:5229 is blocked on a mutex likely owned by task python3:3254.
[ 1358.685543] task:python3 state:D stack:25312 pid:3254 tgid:3254 ppid:3253 task_flags:0x400100 flags:0x00080802
[ 1358.698742] Call Trace:
[ 1358.702281]
[ 1358.705432] __schedule+0xad9/0x1490
[ 1358.710234] ? io_schedule_timeout+0x130/0x130
[ 1358.716018] ? find_held_lock+0x2b/0x80
[ 1358.721121] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.727300] schedule+0xd4/0x260
[ 1358.731717] schedule_timeout+0x103/0x260
[ 1358.737013] ? rcu_is_watching+0x15/0xd0
[ 1358.742214] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.748584] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.754267] ? ww_mutex_lock+0x160/0x160
[ 1358.759467] msleep+0x1b/0x30
[ 1358.763595] napi_set_threaded+0x376/0x410
[ 1358.768991] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.774487] netif_set_threaded+0x254/0x400
[ 1358.779979] ? sysfs_file_ops+0x140/0x140
[ 1358.785275] threaded_store+0x1bb/0x220
[ 1358.790378] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.795872] ? sysfs_file_ops+0x140/0x140
[ 1358.801169] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.806467] ? sysfs_kf_write+0x65/0x160
[ 1358.811666] kernfs_fop_write_iter+0x335/0x510
[ 1358.817450] ? kernfs_ops+0xf0/0xf0
[ 1358.822163] new_sync_write+0x308/0x6f0
[ 1358.827259] ? new_sync_read+0x6f0/0x6f0
[ 1358.832461] ? lock_acquire.part.0+0xbc/0x260
[ 1358.838147] ? ksys_write+0xff/0x200
[ 1358.842962] vfs_write+0x62a/0x980
[ 1358.847576] ksys_write+0xff/0x200
[ 1358.852192] ? __ia32_sys_read+0xc0/0xc0
[ 1358.857393] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.862985] ? rcu_is_watching+0x15/0xd0
[ 1358.868187] do_syscall_64+0xf3/0x5e0
[ 1358.873094] ? exc_page_fault+0x9e/0xb0
[ 1358.878195] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.884663] RIP: 0033:0x7f8663516c5e
[ 1358.889471] RSP: 002b:00007ffce7020950 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.898771] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8663516c5e
[ 1358.907580] RDX: 0000000000000001 RSI: 000055b4d6854760 RDI: 0000000000000007
[ 1358.916388] RBP: 00007ffce7020960 R08: 0000000000000000 R09: 0000000000000000
[ 1358.925199] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f86633b1e80
[ 1358.934006] R13: 000055b4d6854760 R14: 0000000000000007 R15: 00007f8663342480
[ 1358.942821]
[ 1358.946092] \x0aShowing all locks held in the system:
[ 1358.954629] 1 lock held by khungtaskd/184:
[ 1358.960021] #0: ffffffffadcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.971504] 4 locks held by python3/3254:
[ 1358.976799] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.986592] #1: ff1100013df1a088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.997557] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.008720] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.018813] 4 locks held by python3/5229:
[ 1359.024110] #0: ff1100012df913f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.033900] #1: ff11000122a45688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.044861] #2: ff11000120a390f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.056013] #3: ff11000120b51d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.066108]
[ 1359.068573] =============================================\x0a