[ 782.511280] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 782.513705] nipa-hw-worker: [4/40] Retrying drivers/net:napi_threaded.py
[ 782.669378] kselftest: Running tests in drivers/net
[ 989.711904] INFO: task python3:5454 blocked for more than 122 seconds.
[ 989.720128] Not tainted 7.0.0-rc6-mhnr-g441d3384037a #1
[ 989.727380] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.736972] task:python3 state:D stack:25248 pid:5454 tgid:5454 ppid:5453 task_flags:0x400100 flags:0x00080000
[ 989.750177] Call Trace:
[ 989.753718]
[ 989.756871] __schedule+0xad9/0x1490
[ 989.761688] ? io_schedule_timeout+0x130/0x130
[ 989.767473] ? find_held_lock+0x2b/0x80
[ 989.772582] ? __lock_release.isra.0+0x59/0x170
[ 989.778468] schedule+0xd4/0x260
[ 989.782889] schedule_preempt_disabled+0x18/0x30
[ 989.788868] __mutex_lock+0xd76/0x1f20
[ 989.793865] ? threaded_store+0x132/0x220
[ 989.799168] ? ww_mutex_lock+0x160/0x160
[ 989.804369] ? selinux_inode_free_security+0x310/0x310
[ 989.810940] ? cap_capable+0x1ed/0x3f0
[ 989.815947] ? sysfs_file_ops+0x140/0x140
[ 989.821244] ? threaded_store+0x132/0x220
[ 989.826541] threaded_store+0x132/0x220
[ 989.831648] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.837143] ? sysfs_file_ops+0x140/0x140
[ 989.842442] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.847741] ? sysfs_kf_write+0x65/0x160
[ 989.852940] kernfs_fop_write_iter+0x335/0x510
[ 989.858728] ? kernfs_ops+0xf0/0xf0
[ 989.863444] new_sync_write+0x308/0x6f0
[ 989.868548] ? new_sync_read+0x6f0/0x6f0
[ 989.873748] ? lock_acquire.part.0+0xbc/0x260
[ 989.879428] ? ksys_write+0xff/0x200
[ 989.884240] vfs_write+0x62a/0x980
[ 989.888855] ksys_write+0xff/0x200
[ 989.893471] ? __ia32_sys_read+0xc0/0xc0
[ 989.898670] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.904263] ? rcu_is_watching+0x15/0xd0
[ 989.909466] do_syscall_64+0xf3/0x5e0
[ 989.914377] ? exc_page_fault+0x9e/0xb0
[ 989.919479] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.925946] RIP: 0033:0x7f32e844bc5e
[ 989.930757] RSP: 002b:00007ffff5f6aff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.940058] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f32e844bc5e
[ 989.948866] RDX: 0000000000000001 RSI: 000055c412018760 RDI: 0000000000000007
[ 989.957664] RBP: 00007ffff5f6b000 R08: 0000000000000000 R09: 0000000000000000
[ 989.966471] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f32e82e6e80
[ 989.975279] R13: 000055c412018760 R14: 0000000000000007 R15: 00007f32e8b4cb98
[ 989.984087]
[ 989.987379] INFO: task python3:5454 is blocked on a mutex likely owned by task python3:3509.
[ 989.997650] task:python3 state:D stack:25312 pid:3509 tgid:3509 ppid:3508 task_flags:0x400100 flags:0x00080802
[ 990.010855] Call Trace:
[ 990.014396]
[ 990.017545] __schedule+0xad9/0x1490
[ 990.022355] ? io_schedule_timeout+0x130/0x130
[ 990.028142] ? find_held_lock+0x2b/0x80
[ 990.033243] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.039425] schedule+0xd4/0x260
[ 990.043844] schedule_timeout+0x103/0x260
[ 990.049142] ? rcu_is_watching+0x15/0xd0
[ 990.054344] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.060717] ? hrtimers_cpu_dying+0x250/0x250
[ 990.066409] ? ww_mutex_lock+0x160/0x160
[ 990.071610] msleep+0x1b/0x30
[ 990.075740] napi_set_threaded+0x376/0x410
[ 990.081138] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.086631] netif_set_threaded+0x254/0x400
[ 990.092126] ? sysfs_file_ops+0x140/0x140
[ 990.097422] threaded_store+0x1bb/0x220
[ 990.102527] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.108018] ? sysfs_file_ops+0x140/0x140
[ 990.113316] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.118612] ? sysfs_kf_write+0x65/0x160
[ 990.123814] kernfs_fop_write_iter+0x335/0x510
[ 990.129600] ? kernfs_ops+0xf0/0xf0
[ 990.134310] new_sync_write+0x308/0x6f0
[ 990.139412] ? new_sync_read+0x6f0/0x6f0
[ 990.144615] ? lock_acquire.part.0+0xbc/0x260
[ 990.150302] ? ksys_write+0xff/0x200
[ 990.155113] vfs_write+0x62a/0x980
[ 990.159728] ksys_write+0xff/0x200
[ 990.164346] ? __ia32_sys_read+0xc0/0xc0
[ 990.169544] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.175133] ? rcu_is_watching+0x15/0xd0
[ 990.180332] do_syscall_64+0xf3/0x5e0
[ 990.185237] ? irq_exit_rcu+0x39/0xf0
[ 990.190146] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.196606] RIP: 0033:0x7fe624692c5e
[ 990.201415] RSP: 002b:00007ffdf7de4ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.210711] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe624692c5e
[ 990.219517] RDX: 0000000000000001 RSI: 0000563d2addb760 RDI: 0000000000000007
[ 990.228324] RBP: 00007ffdf7de4ae0 R08: 0000000000000000 R09: 0000000000000000
[ 990.237131] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe62452de80
[ 990.245941] R13: 0000563d2addb760 R14: 0000000000000007 R15: 00007fe6244be480
[ 990.254757]
[ 990.258017] \x0aShowing all locks held in the system:
[ 990.266558] 1 lock held by khungtaskd/185:
[ 990.271950] #0: ffffffff936ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.283422] 2 locks held by systemd-journal/804:
[ 990.289402] #0: ff1100180ddb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 990.299289] #1: ffffffff938c9340 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.312117] 4 locks held by python3/3509:
[ 990.317412] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.327203] #1: ff1100015a11a688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.338158] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.349315] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.359417] 4 locks held by python3/5454:
[ 990.364715] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.374506] #1: ff110001ba576288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.385460] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.396616] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.406707]
[ 990.409174] =============================================\x0a
[ 1112.594069] INFO: task python3:5454 blocked for more than 245 seconds.
[ 1112.602221] Not tainted 7.0.0-rc6-mhnr-g441d3384037a #1
[ 1112.609478] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.619065] task:python3 state:D stack:25248 pid:5454 tgid:5454 ppid:5453 task_flags:0x400100 flags:0x00080000
[ 1112.632272] Call Trace:
[ 1112.635808]
[ 1112.638960] __schedule+0xad9/0x1490
[ 1112.643775] ? io_schedule_timeout+0x130/0x130
[ 1112.649559] ? find_held_lock+0x2b/0x80
[ 1112.654665] ? __lock_release.isra.0+0x59/0x170
[ 1112.660553] schedule+0xd4/0x260
[ 1112.664968] schedule_preempt_disabled+0x18/0x30
[ 1112.670949] __mutex_lock+0xd76/0x1f20
[ 1112.675943] ? threaded_store+0x132/0x220
[ 1112.681236] ? ww_mutex_lock+0x160/0x160
[ 1112.686435] ? selinux_inode_free_security+0x310/0x310
[ 1112.693003] ? cap_capable+0x1ed/0x3f0
[ 1112.698010] ? sysfs_file_ops+0x140/0x140
[ 1112.703309] ? threaded_store+0x132/0x220
[ 1112.708609] threaded_store+0x132/0x220
[ 1112.713713] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.719201] ? sysfs_file_ops+0x140/0x140
[ 1112.724499] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.729795] ? sysfs_kf_write+0x65/0x160
[ 1112.734995] kernfs_fop_write_iter+0x335/0x510
[ 1112.740772] ? kernfs_ops+0xf0/0xf0
[ 1112.745484] new_sync_write+0x308/0x6f0
[ 1112.750576] ? new_sync_read+0x6f0/0x6f0
[ 1112.755776] ? lock_acquire.part.0+0xbc/0x260
[ 1112.761461] ? ksys_write+0xff/0x200
[ 1112.766272] vfs_write+0x62a/0x980
[ 1112.770875] ksys_write+0xff/0x200
[ 1112.775488] ? __ia32_sys_read+0xc0/0xc0
[ 1112.780683] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.786271] ? rcu_is_watching+0x15/0xd0
[ 1112.791470] do_syscall_64+0xf3/0x5e0
[ 1112.796366] ? exc_page_fault+0x9e/0xb0
[ 1112.801464] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.807929] RIP: 0033:0x7f32e844bc5e
[ 1112.812739] RSP: 002b:00007ffff5f6aff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.822029] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f32e844bc5e
[ 1112.830834] RDX: 0000000000000001 RSI: 000055c412018760 RDI: 0000000000000007
[ 1112.839638] RBP: 00007ffff5f6b000 R08: 0000000000000000 R09: 0000000000000000
[ 1112.848445] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f32e82e6e80
[ 1112.857251] R13: 000055c412018760 R14: 0000000000000007 R15: 00007f32e8b4cb98
[ 1112.866064]
[ 1112.869345] INFO: task python3:5454 is blocked on a mutex likely owned by task python3:3509.
[ 1112.879618] task:python3 state:D stack:25312 pid:3509 tgid:3509 ppid:3508 task_flags:0x400100 flags:0x00080802
[ 1112.892821] Call Trace:
[ 1112.896358]
[ 1112.899508] __schedule+0xad9/0x1490
[ 1112.904317] ? io_schedule_timeout+0x130/0x130
[ 1112.910098] ? find_held_lock+0x2b/0x80
[ 1112.915198] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.921378] schedule+0xd4/0x260
[ 1112.925796] schedule_timeout+0x103/0x260
[ 1112.931093] ? rcu_is_watching+0x15/0xd0
[ 1112.936290] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.942662] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.948349] ? ww_mutex_lock+0x160/0x160
[ 1112.953542] msleep+0x1b/0x30
[ 1112.957665] napi_set_threaded+0x376/0x410
[ 1112.963057] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.968552] netif_set_threaded+0x254/0x400
[ 1112.974042] ? sysfs_file_ops+0x140/0x140
[ 1112.979338] threaded_store+0x1bb/0x220
[ 1112.984438] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.989931] ? sysfs_file_ops+0x140/0x140
[ 1112.995219] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.000515] ? sysfs_kf_write+0x65/0x160
[ 1113.005716] kernfs_fop_write_iter+0x335/0x510
[ 1113.011502] ? kernfs_ops+0xf0/0xf0
[ 1113.016211] new_sync_write+0x308/0x6f0
[ 1113.021313] ? new_sync_read+0x6f0/0x6f0
[ 1113.026510] ? lock_acquire.part.0+0xbc/0x260
[ 1113.032197] ? ksys_write+0xff/0x200
[ 1113.037000] vfs_write+0x62a/0x980
[ 1113.041615] ksys_write+0xff/0x200
[ 1113.046228] ? __ia32_sys_read+0xc0/0xc0
[ 1113.051426] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.057015] ? rcu_is_watching+0x15/0xd0
[ 1113.062215] do_syscall_64+0xf3/0x5e0
[ 1113.067111] ? irq_exit_rcu+0x39/0xf0
[ 1113.072019] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.078485] RIP: 0033:0x7fe624692c5e
[ 1113.083290] RSP: 002b:00007ffdf7de4ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.092585] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe624692c5e
[ 1113.101392] RDX: 0000000000000001 RSI: 0000563d2addb760 RDI: 0000000000000007
[ 1113.110192] RBP: 00007ffdf7de4ae0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.118990] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe62452de80
[ 1113.127788] R13: 0000563d2addb760 R14: 0000000000000007 R15: 00007fe6244be480
[ 1113.136600]
[ 1113.139864] \x0aShowing all locks held in the system:
[ 1113.148411] 1 lock held by khungtaskd/185:
[ 1113.153806] #0: ffffffff936ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.165286] 4 locks held by python3/3509:
[ 1113.170579] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.180369] #1: ff1100015a11a688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.191333] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.202488] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.212584] 4 locks held by python3/5454:
[ 1113.217879] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.227667] #1: ff110001ba576288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.238627] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.249784] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.259875]
[ 1113.262341] =============================================\x0a
[ 1235.475265] INFO: task python3:5454 blocked for more than 368 seconds.
[ 1235.483414] Not tainted 7.0.0-rc6-mhnr-g441d3384037a #1
[ 1235.490663] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.500252] task:python3 state:D stack:25248 pid:5454 tgid:5454 ppid:5453 task_flags:0x400100 flags:0x00080002
[ 1235.513459] Call Trace:
[ 1235.516998]
[ 1235.520150] __schedule+0xad9/0x1490
[ 1235.524966] ? io_schedule_timeout+0x130/0x130
[ 1235.530750] ? find_held_lock+0x2b/0x80
[ 1235.535857] ? __lock_release.isra.0+0x59/0x170
[ 1235.541739] schedule+0xd4/0x260
[ 1235.546150] schedule_preempt_disabled+0x18/0x30
[ 1235.552121] __mutex_lock+0xd76/0x1f20
[ 1235.557125] ? threaded_store+0x132/0x220
[ 1235.562422] ? ww_mutex_lock+0x160/0x160
[ 1235.567621] ? selinux_inode_free_security+0x310/0x310
[ 1235.574189] ? cap_capable+0x1ed/0x3f0
[ 1235.579196] ? sysfs_file_ops+0x140/0x140
[ 1235.584494] ? threaded_store+0x132/0x220
[ 1235.589787] threaded_store+0x132/0x220
[ 1235.594890] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.600383] ? sysfs_file_ops+0x140/0x140
[ 1235.605678] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.610971] ? sysfs_kf_write+0x65/0x160
[ 1235.616170] kernfs_fop_write_iter+0x335/0x510
[ 1235.621956] ? kernfs_ops+0xf0/0xf0
[ 1235.626659] new_sync_write+0x308/0x6f0
[ 1235.631761] ? new_sync_read+0x6f0/0x6f0
[ 1235.636962] ? lock_acquire.part.0+0xbc/0x260
[ 1235.642647] ? ksys_write+0xff/0x200
[ 1235.647457] vfs_write+0x62a/0x980
[ 1235.652062] ksys_write+0xff/0x200
[ 1235.656674] ? __ia32_sys_read+0xc0/0xc0
[ 1235.661869] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.667457] ? rcu_is_watching+0x15/0xd0
[ 1235.672657] do_syscall_64+0xf3/0x5e0
[ 1235.677562] ? exc_page_fault+0x9e/0xb0
[ 1235.682663] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.689127] RIP: 0033:0x7f32e844bc5e
[ 1235.693939] RSP: 002b:00007ffff5f6aff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.703236] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f32e844bc5e
[ 1235.712034] RDX: 0000000000000001 RSI: 000055c412018760 RDI: 0000000000000007
[ 1235.720838] RBP: 00007ffff5f6b000 R08: 0000000000000000 R09: 0000000000000000
[ 1235.729644] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f32e82e6e80
[ 1235.738453] R13: 000055c412018760 R14: 0000000000000007 R15: 00007f32e8b4cb98
[ 1235.747265]
[ 1235.750532] INFO: task python3:5454 is blocked on a mutex likely owned by task python3:3509.
[ 1235.760792] task:python3 state:D stack:25312 pid:3509 tgid:3509 ppid:3508 task_flags:0x400100 flags:0x00080802
[ 1235.773996] Call Trace:
[ 1235.777533]
[ 1235.780684] __schedule+0xad9/0x1490
[ 1235.785484] ? io_schedule_timeout+0x130/0x130
[ 1235.791270] ? find_held_lock+0x2b/0x80
[ 1235.796370] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.802551] schedule+0xd4/0x260
[ 1235.806969] schedule_timeout+0x103/0x260
[ 1235.812266] ? rcu_is_watching+0x15/0xd0
[ 1235.817464] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.823833] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.829524] ? ww_mutex_lock+0x160/0x160
[ 1235.834722] msleep+0x1b/0x30
[ 1235.838847] napi_set_threaded+0x376/0x410
[ 1235.844241] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.849734] netif_set_threaded+0x254/0x400
[ 1235.855228] ? sysfs_file_ops+0x140/0x140
[ 1235.860522] threaded_store+0x1bb/0x220
[ 1235.865626] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.871116] ? sysfs_file_ops+0x140/0x140
[ 1235.876411] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.881707] ? sysfs_kf_write+0x65/0x160
[ 1235.886907] kernfs_fop_write_iter+0x335/0x510
[ 1235.892685] ? kernfs_ops+0xf0/0xf0
[ 1235.897395] new_sync_write+0x308/0x6f0
[ 1235.902497] ? new_sync_read+0x6f0/0x6f0
[ 1235.907690] ? lock_acquire.part.0+0xbc/0x260
[ 1235.913374] ? ksys_write+0xff/0x200
[ 1235.918185] vfs_write+0x62a/0x980
[ 1235.922801] ksys_write+0xff/0x200
[ 1235.927416] ? __ia32_sys_read+0xc0/0xc0
[ 1235.932605] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.938194] ? rcu_is_watching+0x15/0xd0
[ 1235.943394] do_syscall_64+0xf3/0x5e0
[ 1235.948291] ? irq_exit_rcu+0x39/0xf0
[ 1235.953198] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.959664] RIP: 0033:0x7fe624692c5e
[ 1235.964472] RSP: 002b:00007ffdf7de4ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.973768] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe624692c5e
[ 1235.982574] RDX: 0000000000000001 RSI: 0000563d2addb760 RDI: 0000000000000007
[ 1235.991380] RBP: 00007ffdf7de4ae0 R08: 0000000000000000 R09: 0000000000000000
[ 1236.000187] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe62452de80
[ 1236.008995] R13: 0000563d2addb760 R14: 0000000000000007 R15: 00007fe6244be480
[ 1236.017798]
[ 1236.021063] \x0aShowing all locks held in the system:
[ 1236.029601] 1 lock held by khungtaskd/185:
[ 1236.034995] #0: ffffffff936ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.046476] 4 locks held by python3/3509:
[ 1236.051769] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.061561] #1: ff1100015a11a688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.072523] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.083679] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.093771] 4 locks held by python3/5454:
[ 1236.099066] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.108858] #1: ff110001ba576288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.119819] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.130967] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.141059]
[ 1236.143525] =============================================\x0a
[ 1280.531676] kworker/u129:14 (310) used greatest stack depth: 21688 bytes left
[ 1358.355450] INFO: task python3:5454 blocked for more than 491 seconds.
[ 1358.364157] Not tainted 7.0.0-rc6-mhnr-g441d3384037a #1
[ 1358.371414] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.381003] task:python3 state:D stack:25248 pid:5454 tgid:5454 ppid:5453 task_flags:0x400100 flags:0x00080002
[ 1358.394208] Call Trace:
[ 1358.397748]
[ 1358.400896] __schedule+0xad9/0x1490
[ 1358.405714] ? io_schedule_timeout+0x130/0x130
[ 1358.411498] ? find_held_lock+0x2b/0x80
[ 1358.416601] ? __lock_release.isra.0+0x59/0x170
[ 1358.422484] schedule+0xd4/0x260
[ 1358.426901] schedule_preempt_disabled+0x18/0x30
[ 1358.432881] __mutex_lock+0xd76/0x1f20
[ 1358.437886] ? threaded_store+0x132/0x220
[ 1358.443183] ? ww_mutex_lock+0x160/0x160
[ 1358.448379] ? selinux_inode_free_security+0x310/0x310
[ 1358.454948] ? cap_capable+0x1ed/0x3f0
[ 1358.459953] ? sysfs_file_ops+0x140/0x140
[ 1358.465251] ? threaded_store+0x132/0x220
[ 1358.470545] threaded_store+0x132/0x220
[ 1358.475648] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.481139] ? sysfs_file_ops+0x140/0x140
[ 1358.486436] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.491731] ? sysfs_kf_write+0x65/0x160
[ 1358.496947] kernfs_fop_write_iter+0x335/0x510
[ 1358.502757] ? kernfs_ops+0xf0/0xf0
[ 1358.507472] new_sync_write+0x308/0x6f0
[ 1358.512579] ? new_sync_read+0x6f0/0x6f0
[ 1358.517781] ? lock_acquire.part.0+0xbc/0x260
[ 1358.523468] ? ksys_write+0xff/0x200
[ 1358.528287] vfs_write+0x62a/0x980
[ 1358.532897] ksys_write+0xff/0x200
[ 1358.537511] ? __ia32_sys_read+0xc0/0xc0
[ 1358.542704] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.548293] ? rcu_is_watching+0x15/0xd0
[ 1358.553497] do_syscall_64+0xf3/0x5e0
[ 1358.558409] ? exc_page_fault+0x9e/0xb0
[ 1358.563510] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.569978] RIP: 0033:0x7f32e844bc5e
[ 1358.574788] RSP: 002b:00007ffff5f6aff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.584092] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f32e844bc5e
[ 1358.592905] RDX: 0000000000000001 RSI: 000055c412018760 RDI: 0000000000000007
[ 1358.601714] RBP: 00007ffff5f6b000 R08: 0000000000000000 R09: 0000000000000000
[ 1358.610523] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f32e82e6e80
[ 1358.619334] R13: 000055c412018760 R14: 0000000000000007 R15: 00007f32e8b4cb98
[ 1358.628152]
[ 1358.631500] INFO: task python3:5454 is blocked on a mutex likely owned by task python3:3509.
[ 1358.641777] task:python3 state:D stack:25312 pid:3509 tgid:3509 ppid:3508 task_flags:0x400100 flags:0x00080802
[ 1358.654983] Call Trace:
[ 1358.658526]
[ 1358.661667] __schedule+0xad9/0x1490
[ 1358.666482] ? io_schedule_timeout+0x130/0x130
[ 1358.672267] ? find_held_lock+0x2b/0x80
[ 1358.677442] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.683622] schedule+0xd4/0x260
[ 1358.688041] schedule_timeout+0x103/0x260
[ 1358.693341] ? rcu_is_watching+0x15/0xd0
[ 1358.698540] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.704907] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.710597] ? ww_mutex_lock+0x160/0x160
[ 1358.715796] msleep+0x1b/0x30
[ 1358.719922] napi_set_threaded+0x376/0x410
[ 1358.725318] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.730816] netif_set_threaded+0x254/0x400
[ 1358.736310] ? sysfs_file_ops+0x140/0x140
[ 1358.741609] threaded_store+0x1bb/0x220
[ 1358.746715] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.752209] ? sysfs_file_ops+0x140/0x140
[ 1358.757509] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.762807] ? sysfs_kf_write+0x65/0x160
[ 1358.768010] kernfs_fop_write_iter+0x335/0x510
[ 1358.773796] ? kernfs_ops+0xf0/0xf0
[ 1358.778509] new_sync_write+0x308/0x6f0
[ 1358.783604] ? new_sync_read+0x6f0/0x6f0
[ 1358.788808] ? lock_acquire.part.0+0xbc/0x260
[ 1358.794562] ? ksys_write+0xff/0x200
[ 1358.799375] vfs_write+0x62a/0x980
[ 1358.803991] ksys_write+0xff/0x200
[ 1358.808607] ? __ia32_sys_read+0xc0/0xc0
[ 1358.813807] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.819396] ? rcu_is_watching+0x15/0xd0
[ 1358.824598] do_syscall_64+0xf3/0x5e0
[ 1358.829506] ? irq_exit_rcu+0x39/0xf0
[ 1358.834414] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.840883] RIP: 0033:0x7fe624692c5e
[ 1358.845691] RSP: 002b:00007ffdf7de4ad0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.854991] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe624692c5e
[ 1358.863803] RDX: 0000000000000001 RSI: 0000563d2addb760 RDI: 0000000000000007
[ 1358.872615] RBP: 00007ffdf7de4ae0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.881426] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe62452de80
[ 1358.890238] R13: 0000563d2addb760 R14: 0000000000000007 R15: 00007fe6244be480
[ 1358.899053]
[ 1358.902327] \x0aShowing all locks held in the system:
[ 1358.910869] 1 lock held by khungtaskd/185:
[ 1358.916256] #0: ffffffff936ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.927742] 4 locks held by python3/3509:
[ 1358.933039] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.942823] #1: ff1100015a11a688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.953789] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.964948] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.975054] 4 locks held by python3/5454:
[ 1358.980349] #0: ff1100012fb193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.990143] #1: ff110001ba576288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.001107] #2: ff110001227130f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.012268] #3: ff11000122799d38 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.022363]
[ 1359.024831] =============================================\x0a