[ 759.340876] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 759.343335] nipa-hw-worker: [4/39] Retrying drivers/net:napi_threaded.py
[ 759.505369] kselftest: Running tests in drivers/net
[ 989.741675] INFO: task python3:5003 blocked for more than 122 seconds.
[ 989.749907] Not tainted 7.0.0-rc5-yxnf-g011c8d1f0e5e #1
[ 989.757158] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.766747] task:python3 state:D stack:25312 pid:5003 tgid:5003 ppid:5002 task_flags:0x400100 flags:0x00080000
[ 989.779951] Call Trace:
[ 989.783488]
[ 989.786638] __schedule+0xad9/0x1490
[ 989.791456] ? io_schedule_timeout+0x130/0x130
[ 989.797240] ? find_held_lock+0x2b/0x80
[ 989.802343] ? __lock_release.isra.0+0x59/0x170
[ 989.808229] schedule+0xd4/0x260
[ 989.812646] schedule_preempt_disabled+0x18/0x30
[ 989.818624] __mutex_lock+0xd76/0x1f20
[ 989.823627] ? threaded_store+0x132/0x220
[ 989.828926] ? ww_mutex_lock+0x160/0x160
[ 989.834126] ? selinux_inode_free_security+0x310/0x310
[ 989.840696] ? cap_capable+0x1ed/0x3f0
[ 989.845702] ? sysfs_file_ops+0x140/0x140
[ 989.850999] ? threaded_store+0x132/0x220
[ 989.856295] threaded_store+0x132/0x220
[ 989.861395] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.866889] ? sysfs_file_ops+0x140/0x140
[ 989.872186] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.877483] ? sysfs_kf_write+0x65/0x160
[ 989.882683] kernfs_fop_write_iter+0x335/0x510
[ 989.888460] ? kernfs_ops+0xf0/0xf0
[ 989.893169] new_sync_write+0x308/0x6f0
[ 989.898273] ? new_sync_read+0x6f0/0x6f0
[ 989.903465] ? lock_acquire.part.0+0xbc/0x260
[ 989.909149] ? ksys_write+0xff/0x200
[ 989.913961] vfs_write+0x62a/0x980
[ 989.918576] ksys_write+0xff/0x200
[ 989.923181] ? __ia32_sys_read+0xc0/0xc0
[ 989.928377] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.933965] ? rcu_is_watching+0x15/0xd0
[ 989.939167] do_syscall_64+0xf3/0x5e0
[ 989.944074] ? exc_page_fault+0x9e/0xb0
[ 989.949175] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.955642] RIP: 0033:0x7f53d1401c5e
[ 989.960450] RSP: 002b:00007ffd24c42120 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.969751] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f53d1401c5e
[ 989.978559] RDX: 0000000000000001 RSI: 0000556367da0760 RDI: 0000000000000007
[ 989.987357] RBP: 00007ffd24c42130 R08: 0000000000000000 R09: 0000000000000000
[ 989.996161] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f53d129ce80
[ 990.004970] R13: 0000556367da0760 R14: 0000000000000007 R15: 00007f53d1b02b98
[ 990.013783]
[ 990.017057] INFO: task python3:5003 is blocked on a mutex likely owned by task python3:3080.
[ 990.027322] task:python3 state:D stack:25312 pid:3080 tgid:3080 ppid:3079 task_flags:0x400100 flags:0x00080802
[ 990.040529] Call Trace:
[ 990.044066]
[ 990.047213] __schedule+0xad9/0x1490
[ 990.052023] ? io_schedule_timeout+0x130/0x130
[ 990.057805] ? find_held_lock+0x2b/0x80
[ 990.062906] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.069083] schedule+0xd4/0x260
[ 990.073503] schedule_timeout+0x103/0x260
[ 990.078797] ? rcu_is_watching+0x15/0xd0
[ 990.083994] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.090364] ? hrtimers_cpu_dying+0x250/0x250
[ 990.096053] ? ww_mutex_lock+0x160/0x160
[ 990.101250] msleep+0x1b/0x30
[ 990.105377] napi_set_threaded+0x376/0x410
[ 990.110772] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.116268] netif_set_threaded+0x254/0x400
[ 990.121759] ? sysfs_file_ops+0x140/0x140
[ 990.127056] threaded_store+0x1bb/0x220
[ 990.132155] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.137647] ? sysfs_file_ops+0x140/0x140
[ 990.142934] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.148227] ? sysfs_kf_write+0x65/0x160
[ 990.153425] kernfs_fop_write_iter+0x335/0x510
[ 990.159207] ? kernfs_ops+0xf0/0xf0
[ 990.163916] new_sync_write+0x308/0x6f0
[ 990.169020] ? new_sync_read+0x6f0/0x6f0
[ 990.174219] ? lock_acquire.part.0+0xbc/0x260
[ 990.179903] ? ksys_write+0xff/0x200
[ 990.184716] vfs_write+0x62a/0x980
[ 990.189329] ksys_write+0xff/0x200
[ 990.193941] ? __ia32_sys_read+0xc0/0xc0
[ 990.199138] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.204728] ? rcu_is_watching+0x15/0xd0
[ 990.209925] do_syscall_64+0xf3/0x5e0
[ 990.214828] ? irq_exit_rcu+0x39/0xf0
[ 990.219734] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.226201] RIP: 0033:0x7f4d1b4adc5e
[ 990.231007] RSP: 002b:00007ffe398042f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.240301] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4d1b4adc5e
[ 990.249108] RDX: 0000000000000001 RSI: 000055d075965760 RDI: 0000000000000007
[ 990.257908] RBP: 00007ffe39804300 R08: 0000000000000000 R09: 0000000000000000
[ 990.266708] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4d1b348e80
[ 990.275515] R13: 000055d075965760 R14: 0000000000000007 R15: 00007f4d1b2d6480
[ 990.284334]
[ 990.287591] \x0aShowing all locks held in the system:
[ 990.296128] 1 lock held by khungtaskd/185:
[ 990.301526] #0: ffffffffac8ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.313007] 4 locks held by python3/3080:
[ 990.318302] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.328095] #1: ff1100013c2de888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.339058] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.350215] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.360308] 4 locks held by python3/5003:
[ 990.365604] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.375394] #1: ff11000182c05688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.386360] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.397530] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.407614]
[ 990.410078] =============================================\x0a
[ 1112.623502] INFO: task python3:5003 blocked for more than 245 seconds.
[ 1112.631652] Not tainted 7.0.0-rc5-yxnf-g011c8d1f0e5e #1
[ 1112.638901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.648490] task:python3 state:D stack:25312 pid:5003 tgid:5003 ppid:5002 task_flags:0x400100 flags:0x00080000
[ 1112.661696] Call Trace:
[ 1112.665234]
[ 1112.668383] __schedule+0xad9/0x1490
[ 1112.673198] ? io_schedule_timeout+0x130/0x130
[ 1112.678983] ? find_held_lock+0x2b/0x80
[ 1112.684089] ? __lock_release.isra.0+0x59/0x170
[ 1112.689973] schedule+0xd4/0x260
[ 1112.694383] schedule_preempt_disabled+0x18/0x30
[ 1112.700364] __mutex_lock+0xd76/0x1f20
[ 1112.705367] ? threaded_store+0x132/0x220
[ 1112.710666] ? ww_mutex_lock+0x160/0x160
[ 1112.715865] ? selinux_inode_free_security+0x310/0x310
[ 1112.722435] ? cap_capable+0x1ed/0x3f0
[ 1112.727442] ? sysfs_file_ops+0x140/0x140
[ 1112.732742] ? threaded_store+0x132/0x220
[ 1112.738103] threaded_store+0x132/0x220
[ 1112.743204] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.748697] ? sysfs_file_ops+0x140/0x140
[ 1112.753995] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.759293] ? sysfs_kf_write+0x65/0x160
[ 1112.764491] kernfs_fop_write_iter+0x335/0x510
[ 1112.770276] ? kernfs_ops+0xf0/0xf0
[ 1112.774981] new_sync_write+0x308/0x6f0
[ 1112.780086] ? new_sync_read+0x6f0/0x6f0
[ 1112.785284] ? lock_acquire.part.0+0xbc/0x260
[ 1112.790967] ? ksys_write+0xff/0x200
[ 1112.795780] vfs_write+0x62a/0x980
[ 1112.800392] ksys_write+0xff/0x200
[ 1112.805006] ? __ia32_sys_read+0xc0/0xc0
[ 1112.810205] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.815794] ? rcu_is_watching+0x15/0xd0
[ 1112.820997] do_syscall_64+0xf3/0x5e0
[ 1112.825904] ? exc_page_fault+0x9e/0xb0
[ 1112.831003] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.837472] RIP: 0033:0x7f53d1401c5e
[ 1112.842284] RSP: 002b:00007ffd24c42120 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.851582] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f53d1401c5e
[ 1112.860392] RDX: 0000000000000001 RSI: 0000556367da0760 RDI: 0000000000000007
[ 1112.869200] RBP: 00007ffd24c42130 R08: 0000000000000000 R09: 0000000000000000
[ 1112.877998] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f53d129ce80
[ 1112.886807] R13: 0000556367da0760 R14: 0000000000000007 R15: 00007f53d1b02b98
[ 1112.895621]
[ 1112.898899] INFO: task python3:5003 is blocked on a mutex likely owned by task python3:3080.
[ 1112.909170] task:python3 state:D stack:25312 pid:3080 tgid:3080 ppid:3079 task_flags:0x400100 flags:0x00080802
[ 1112.922371] Call Trace:
[ 1112.925910]
[ 1112.929058] __schedule+0xad9/0x1490
[ 1112.933866] ? io_schedule_timeout+0x130/0x130
[ 1112.939648] ? find_held_lock+0x2b/0x80
[ 1112.944749] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.950928] schedule+0xd4/0x260
[ 1112.955349] schedule_timeout+0x103/0x260
[ 1112.960646] ? rcu_is_watching+0x15/0xd0
[ 1112.965846] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.972215] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.977906] ? ww_mutex_lock+0x160/0x160
[ 1112.983106] msleep+0x1b/0x30
[ 1112.987225] napi_set_threaded+0x376/0x410
[ 1112.992622] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.998184] netif_set_threaded+0x254/0x400
[ 1113.003678] ? sysfs_file_ops+0x140/0x140
[ 1113.008973] threaded_store+0x1bb/0x220
[ 1113.014075] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.019567] ? sysfs_file_ops+0x140/0x140
[ 1113.024854] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.030147] ? sysfs_kf_write+0x65/0x160
[ 1113.035348] kernfs_fop_write_iter+0x335/0x510
[ 1113.041131] ? kernfs_ops+0xf0/0xf0
[ 1113.045908] new_sync_write+0x308/0x6f0
[ 1113.051008] ? new_sync_read+0x6f0/0x6f0
[ 1113.056207] ? lock_acquire.part.0+0xbc/0x260
[ 1113.061892] ? ksys_write+0xff/0x200
[ 1113.066703] vfs_write+0x62a/0x980
[ 1113.071320] ksys_write+0xff/0x200
[ 1113.075936] ? __ia32_sys_read+0xc0/0xc0
[ 1113.081134] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.086723] ? rcu_is_watching+0x15/0xd0
[ 1113.091932] do_syscall_64+0xf3/0x5e0
[ 1113.096836] ? irq_exit_rcu+0x39/0xf0
[ 1113.101745] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.108201] RIP: 0033:0x7f4d1b4adc5e
[ 1113.113010] RSP: 002b:00007ffe398042f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.122304] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4d1b4adc5e
[ 1113.131110] RDX: 0000000000000001 RSI: 000055d075965760 RDI: 0000000000000007
[ 1113.139919] RBP: 00007ffe39804300 R08: 0000000000000000 R09: 0000000000000000
[ 1113.148796] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4d1b348e80
[ 1113.157605] R13: 000055d075965760 R14: 0000000000000007 R15: 00007f4d1b2d6480
[ 1113.166418]
[ 1113.169680] \x0aShowing all locks held in the system:
[ 1113.178218] 1 lock held by khungtaskd/185:
[ 1113.183612] #0: ffffffffac8ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.195092] 4 locks held by python3/3080:
[ 1113.200387] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.210179] #1: ff1100013c2de888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.221142] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.232299] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.242396] 4 locks held by python3/5003:
[ 1113.247688] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.257479] #1: ff11000182c05688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.268431] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.279585] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.289675]
[ 1113.292144] =============================================\x0a
[ 1235.504915] INFO: task python3:5003 blocked for more than 368 seconds.
[ 1235.513053] Not tainted 7.0.0-rc5-yxnf-g011c8d1f0e5e #1
[ 1235.520305] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.529893] task:python3 state:D stack:25312 pid:5003 tgid:5003 ppid:5002 task_flags:0x400100 flags:0x00080002
[ 1235.543099] Call Trace:
[ 1235.546638]
[ 1235.549787] __schedule+0xad9/0x1490
[ 1235.554602] ? io_schedule_timeout+0x130/0x130
[ 1235.560379] ? find_held_lock+0x2b/0x80
[ 1235.565482] ? __lock_release.isra.0+0x59/0x170
[ 1235.571369] schedule+0xd4/0x260
[ 1235.575785] schedule_preempt_disabled+0x18/0x30
[ 1235.581763] __mutex_lock+0xd76/0x1f20
[ 1235.586768] ? threaded_store+0x132/0x220
[ 1235.592066] ? ww_mutex_lock+0x160/0x160
[ 1235.597262] ? selinux_inode_free_security+0x310/0x310
[ 1235.603831] ? cap_capable+0x1ed/0x3f0
[ 1235.608835] ? sysfs_file_ops+0x140/0x140
[ 1235.614131] ? threaded_store+0x132/0x220
[ 1235.619426] threaded_store+0x132/0x220
[ 1235.624525] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.630020] ? sysfs_file_ops+0x140/0x140
[ 1235.635317] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.640613] ? sysfs_kf_write+0x65/0x160
[ 1235.645810] kernfs_fop_write_iter+0x335/0x510
[ 1235.651597] ? kernfs_ops+0xf0/0xf0
[ 1235.656306] new_sync_write+0x308/0x6f0
[ 1235.661410] ? new_sync_read+0x6f0/0x6f0
[ 1235.666600] ? lock_acquire.part.0+0xbc/0x260
[ 1235.672287] ? ksys_write+0xff/0x200
[ 1235.677098] vfs_write+0x62a/0x980
[ 1235.681714] ksys_write+0xff/0x200
[ 1235.686328] ? __ia32_sys_read+0xc0/0xc0
[ 1235.691523] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.697110] ? rcu_is_watching+0x15/0xd0
[ 1235.702312] do_syscall_64+0xf3/0x5e0
[ 1235.707217] ? exc_page_fault+0x9e/0xb0
[ 1235.712317] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.718783] RIP: 0033:0x7f53d1401c5e
[ 1235.723591] RSP: 002b:00007ffd24c42120 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.732886] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f53d1401c5e
[ 1235.741693] RDX: 0000000000000001 RSI: 0000556367da0760 RDI: 0000000000000007
[ 1235.750498] RBP: 00007ffd24c42130 R08: 0000000000000000 R09: 0000000000000000
[ 1235.759303] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f53d129ce80
[ 1235.768110] R13: 0000556367da0760 R14: 0000000000000007 R15: 00007f53d1b02b98
[ 1235.776923]
[ 1235.780189] INFO: task python3:5003 is blocked on a mutex likely owned by task python3:3080.
[ 1235.790463] task:python3 state:D stack:25312 pid:3080 tgid:3080 ppid:3079 task_flags:0x400100 flags:0x00080802
[ 1235.803666] Call Trace:
[ 1235.807205]
[ 1235.810354] __schedule+0xad9/0x1490
[ 1235.815163] ? io_schedule_timeout+0x130/0x130
[ 1235.820944] ? find_held_lock+0x2b/0x80
[ 1235.826047] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.832225] schedule+0xd4/0x260
[ 1235.836642] schedule_timeout+0x103/0x260
[ 1235.841939] ? rcu_is_watching+0x15/0xd0
[ 1235.847137] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.853507] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.859196] ? ww_mutex_lock+0x160/0x160
[ 1235.864393] msleep+0x1b/0x30
[ 1235.868516] napi_set_threaded+0x376/0x410
[ 1235.873910] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.879406] netif_set_threaded+0x254/0x400
[ 1235.884897] ? sysfs_file_ops+0x140/0x140
[ 1235.890190] threaded_store+0x1bb/0x220
[ 1235.895292] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.900781] ? sysfs_file_ops+0x140/0x140
[ 1235.906076] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.911373] ? sysfs_kf_write+0x65/0x160
[ 1235.916571] kernfs_fop_write_iter+0x335/0x510
[ 1235.922354] ? kernfs_ops+0xf0/0xf0
[ 1235.927065] new_sync_write+0x308/0x6f0
[ 1235.932169] ? new_sync_read+0x6f0/0x6f0
[ 1235.937368] ? lock_acquire.part.0+0xbc/0x260
[ 1235.943051] ? ksys_write+0xff/0x200
[ 1235.947860] vfs_write+0x62a/0x980
[ 1235.952471] ksys_write+0xff/0x200
[ 1235.957085] ? __ia32_sys_read+0xc0/0xc0
[ 1235.962284] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.967874] ? rcu_is_watching+0x15/0xd0
[ 1235.973075] do_syscall_64+0xf3/0x5e0
[ 1235.977981] ? irq_exit_rcu+0x39/0xf0
[ 1235.982879] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.989345] RIP: 0033:0x7f4d1b4adc5e
[ 1235.994151] RSP: 002b:00007ffe398042f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.003447] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4d1b4adc5e
[ 1236.012254] RDX: 0000000000000001 RSI: 000055d075965760 RDI: 0000000000000007
[ 1236.021052] RBP: 00007ffe39804300 R08: 0000000000000000 R09: 0000000000000000
[ 1236.029860] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4d1b348e80
[ 1236.038668] R13: 000055d075965760 R14: 0000000000000007 R15: 00007f4d1b2d6480
[ 1236.047471]
[ 1236.050737] \x0aShowing all locks held in the system:
[ 1236.059265] 1 lock held by khungtaskd/185:
[ 1236.064655] #0: ffffffffac8ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.076133] 4 locks held by python3/3080:
[ 1236.081428] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.091219] #1: ff1100013c2de888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.102175] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.113330] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.123423] 4 locks held by python3/5003:
[ 1236.128715] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.138503] #1: ff11000182c05688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.149463] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.160619] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.170714]
[ 1236.173180] =============================================\x0a
[ 1358.385377] INFO: task python3:5003 blocked for more than 491 seconds.
[ 1358.393525] Not tainted 7.0.0-rc5-yxnf-g011c8d1f0e5e #1
[ 1358.400778] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.410368] task:python3 state:D stack:25312 pid:5003 tgid:5003 ppid:5002 task_flags:0x400100 flags:0x00080002
[ 1358.423572] Call Trace:
[ 1358.427113]
[ 1358.430263] __schedule+0xad9/0x1490
[ 1358.435080] ? io_schedule_timeout+0x130/0x130
[ 1358.440867] ? find_held_lock+0x2b/0x80
[ 1358.445976] ? __lock_release.isra.0+0x59/0x170
[ 1358.451860] schedule+0xd4/0x260
[ 1358.456279] schedule_preempt_disabled+0x18/0x30
[ 1358.462257] __mutex_lock+0xd76/0x1f20
[ 1358.467264] ? threaded_store+0x132/0x220
[ 1358.472563] ? ww_mutex_lock+0x160/0x160
[ 1358.477763] ? selinux_inode_free_security+0x310/0x310
[ 1358.484333] ? cap_capable+0x1ed/0x3f0
[ 1358.489343] ? sysfs_file_ops+0x140/0x140
[ 1358.494643] ? threaded_store+0x132/0x220
[ 1358.499939] threaded_store+0x132/0x220
[ 1358.505045] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.510539] ? sysfs_file_ops+0x140/0x140
[ 1358.515836] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.521131] ? sysfs_kf_write+0x65/0x160
[ 1358.526333] kernfs_fop_write_iter+0x335/0x510
[ 1358.532119] ? kernfs_ops+0xf0/0xf0
[ 1358.536828] new_sync_write+0x308/0x6f0
[ 1358.541932] ? new_sync_read+0x6f0/0x6f0
[ 1358.547134] ? lock_acquire.part.0+0xbc/0x260
[ 1358.552822] ? ksys_write+0xff/0x200
[ 1358.557628] vfs_write+0x62a/0x980
[ 1358.562242] ksys_write+0xff/0x200
[ 1358.566857] ? __ia32_sys_read+0xc0/0xc0
[ 1358.572054] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.577646] ? rcu_is_watching+0x15/0xd0
[ 1358.582847] do_syscall_64+0xf3/0x5e0
[ 1358.587753] ? exc_page_fault+0x9e/0xb0
[ 1358.592857] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.599328] RIP: 0033:0x7f53d1401c5e
[ 1358.604139] RSP: 002b:00007ffd24c42120 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.613438] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f53d1401c5e
[ 1358.622249] RDX: 0000000000000001 RSI: 0000556367da0760 RDI: 0000000000000007
[ 1358.631056] RBP: 00007ffd24c42130 R08: 0000000000000000 R09: 0000000000000000
[ 1358.639865] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f53d129ce80
[ 1358.648673] R13: 0000556367da0760 R14: 0000000000000007 R15: 00007f53d1b02b98
[ 1358.657489]
[ 1358.660754] INFO: task python3:5003 is blocked on a mutex likely owned by task python3:3080.
[ 1358.671027] task:python3 state:D stack:25312 pid:3080 tgid:3080 ppid:3079 task_flags:0x400100 flags:0x00080802
[ 1358.684233] Call Trace:
[ 1358.687772]
[ 1358.690923] __schedule+0xad9/0x1490
[ 1358.695737] ? io_schedule_timeout+0x130/0x130
[ 1358.701522] ? find_held_lock+0x2b/0x80
[ 1358.706624] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.712793] schedule+0xd4/0x260
[ 1358.717214] schedule_timeout+0x103/0x260
[ 1358.722512] ? rcu_is_watching+0x15/0xd0
[ 1358.727705] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.734076] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.739769] ? ww_mutex_lock+0x160/0x160
[ 1358.744970] msleep+0x1b/0x30
[ 1358.749096] napi_set_threaded+0x376/0x410
[ 1358.754495] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.759989] netif_set_threaded+0x254/0x400
[ 1358.765483] ? sysfs_file_ops+0x140/0x140
[ 1358.770783] threaded_store+0x1bb/0x220
[ 1358.775886] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.781381] ? sysfs_file_ops+0x140/0x140
[ 1358.786679] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.791974] ? sysfs_kf_write+0x65/0x160
[ 1358.797174] kernfs_fop_write_iter+0x335/0x510
[ 1358.802952] ? kernfs_ops+0xf0/0xf0
[ 1358.807664] new_sync_write+0x308/0x6f0
[ 1358.812765] ? new_sync_read+0x6f0/0x6f0
[ 1358.817967] ? lock_acquire.part.0+0xbc/0x260
[ 1358.823653] ? ksys_write+0xff/0x200
[ 1358.828468] vfs_write+0x62a/0x980
[ 1358.833083] ksys_write+0xff/0x200
[ 1358.837686] ? __ia32_sys_read+0xc0/0xc0
[ 1358.842886] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.848468] ? rcu_is_watching+0x15/0xd0
[ 1358.853667] do_syscall_64+0xf3/0x5e0
[ 1358.858577] ? irq_exit_rcu+0x39/0xf0
[ 1358.863487] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.869959] RIP: 0033:0x7f4d1b4adc5e
[ 1358.874768] RSP: 002b:00007ffe398042f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.884056] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4d1b4adc5e
[ 1358.892857] RDX: 0000000000000001 RSI: 000055d075965760 RDI: 0000000000000007
[ 1358.901664] RBP: 00007ffe39804300 R08: 0000000000000000 R09: 0000000000000000
[ 1358.910472] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4d1b348e80
[ 1358.919281] R13: 000055d075965760 R14: 0000000000000007 R15: 00007f4d1b2d6480
[ 1358.928092]
[ 1358.931365] \x0aShowing all locks held in the system:
[ 1358.939902] 1 lock held by khungtaskd/185:
[ 1358.945295] #0: ffffffffac8ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.956780] 4 locks held by python3/3080:
[ 1358.962074] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.971865] #1: ff1100013c2de888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.982829] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.993986] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.004081] 4 locks held by python3/5003:
[ 1359.009375] #0: ff1100012d0f93f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.019164] #1: ff11000182c05688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.030125] #2: ff1100011fb3f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.041274] #3: ff1100011fb81d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.051360]
[ 1359.053826] =============================================\x0a