[ 760.222425] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 760.225348] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 760.379133] kselftest: Running tests in drivers/net
[ 989.711282] INFO: task python3:5370 blocked for more than 122 seconds.
[ 989.719508] Not tainted 7.1.0-rc7-gnws-g21ca060c075b #1
[ 989.727399] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.736990] task:python3 state:D stack:25672 pid:5370 tgid:5370 ppid:5369 task_flags:0x400100 flags:0x00080000
[ 989.750198] Call Trace:
[ 989.753736]
[ 989.756889] __schedule+0xcc3/0x19c0
[ 989.761706] ? io_schedule_timeout+0x140/0x140
[ 989.767490] ? find_held_lock+0x2b/0x80
[ 989.772594] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.778479] schedule+0xd8/0x260
[ 989.782899] schedule_preempt_disabled+0x18/0x30
[ 989.788877] __mutex_lock+0x6af/0x2010
[ 989.793882] ? threaded_store+0x132/0x220
[ 989.799172] ? ww_mutex_lock+0x160/0x160
[ 989.804370] ? selinux_inode_free_security+0x330/0x330
[ 989.810934] ? cap_capable+0x1d6/0x3e0
[ 989.815945] ? threaded_store+0x132/0x220
[ 989.821240] threaded_store+0x132/0x220
[ 989.826343] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.831642] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.836943] ? sysfs_kf_write+0x65/0x160
[ 989.842143] ? sysfs_file_ops+0x140/0x140
[ 989.847440] kernfs_fop_write_iter+0x335/0x510
[ 989.853225] ? kernfs_ops+0xf0/0xf0
[ 989.857931] new_sync_write+0x333/0x750
[ 989.863036] ? __lock_acquire+0x508/0xc10
[ 989.868325] ? new_sync_read+0x740/0x740
[ 989.873520] ? lock_acquire.part.0+0xbc/0x260
[ 989.879208] ? ksys_write+0x116/0x250
[ 989.884116] vfs_write+0x657/0x9b0
[ 989.888723] ksys_write+0x116/0x250
[ 989.893441] ? __ia32_sys_read+0xc0/0xc0
[ 989.898646] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.904239] ? rcu_is_watching+0x15/0xd0
[ 989.909449] do_syscall_64+0xf3/0x650
[ 989.914360] ? trace_hardirqs_off+0xd/0x30
[ 989.919761] ? exc_page_fault+0xda/0xf0
[ 989.924864] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.931338] RIP: 0033:0x7f930763864e
[ 989.936150] RSP: 002b:00007ffe28f407d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.945454] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f930763864e
[ 989.954264] RDX: 0000000000000001 RSI: 0000560d79773bf0 RDI: 0000000000000007
[ 989.963077] RBP: 00007ffe28f407e0 R08: 0000000000000000 R09: 0000000000000000
[ 989.971889] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f93074b1390
[ 989.980704] R13: 0000560d79773bf0 R14: 0000000000000007 R15: 00007f9307445120
[ 989.989532]
[ 989.992849] INFO: task python3:5370 is blocked on a mutex likely owned by task python3:3366.
[ 990.003126] task:python3 state:D stack:25672 pid:3366 tgid:3366 ppid:3365 task_flags:0x400100 flags:0x00080802
[ 990.016341] Call Trace:
[ 990.019883]
[ 990.023039] __schedule+0xcc3/0x19c0
[ 990.027857] ? io_schedule_timeout+0x140/0x140
[ 990.033645] ? find_held_lock+0x2b/0x80
[ 990.038751] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.044937] schedule+0xd8/0x260
[ 990.049362] schedule_timeout+0x13f/0x290
[ 990.054661] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.061033] ? trace_contention_end+0xb3/0x1a0
[ 990.066824] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.072517] ? ww_mutex_lock+0x160/0x160
[ 990.077723] msleep+0x1b/0x30
[ 990.081850] napi_set_threaded+0x3a5/0x440
[ 990.087251] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.092751] netif_set_threaded+0x257/0x400
[ 990.098250] threaded_store+0x1b8/0x220
[ 990.103356] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.108657] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.113959] ? sysfs_kf_write+0x65/0x160
[ 990.119165] ? sysfs_file_ops+0x140/0x140
[ 990.124463] kernfs_fop_write_iter+0x335/0x510
[ 990.130255] ? kernfs_ops+0xf0/0xf0
[ 990.134971] new_sync_write+0x333/0x750
[ 990.140080] ? __lock_acquire+0x508/0xc10
[ 990.145382] ? new_sync_read+0x740/0x740
[ 990.150585] ? lock_acquire.part.0+0xbc/0x260
[ 990.156277] ? ksys_write+0x116/0x250
[ 990.161194] vfs_write+0x657/0x9b0
[ 990.165811] ksys_write+0x116/0x250
[ 990.170525] ? __ia32_sys_read+0xc0/0xc0
[ 990.175726] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.181311] ? rcu_is_watching+0x15/0xd0
[ 990.186515] do_syscall_64+0xf3/0x650
[ 990.191424] ? trace_hardirqs_off+0xd/0x30
[ 990.196823] ? exc_page_fault+0xda/0xf0
[ 990.201930] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.208401] RIP: 0033:0x7fab0446664e
[ 990.213211] RSP: 002b:00007ffcee987e70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.222513] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fab0446664e
[ 990.231330] RDX: 0000000000000001 RSI: 000055b701b3dbf0 RDI: 0000000000000007
[ 990.240145] RBP: 00007ffcee987e80 R08: 0000000000000000 R09: 0000000000000000
[ 990.248960] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fab042df390
[ 990.257771] R13: 000055b701b3dbf0 R14: 0000000000000007 R15: 00007fab04275120
[ 990.266597]
[ 990.269862] \x0aShowing all locks held in the system:
[ 990.278386] 2 locks held by systemd/1:
[ 990.283391] #0: ff110001311093e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.293281] #1: ff110001a064dc80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.304274] 1 lock held by khungtaskd/185:
[ 990.309662] #0: ffffffff8ccfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.321159] 4 locks held by python3/3366:
[ 990.326457] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.336355] #1: ff110001406d0880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.347324] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.358491] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.368598] 4 locks held by python3/5370:
[ 990.373898] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.383789] #1: ff1100011a461a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.394751] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.405916] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.416010]
[ 990.418480] =============================================\x0a
[ 1112.591457] INFO: task python3:5370 blocked for more than 245 seconds.
[ 1112.599607] Not tainted 7.1.0-rc7-gnws-g21ca060c075b #1
[ 1112.606858] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.616447] task:python3 state:D stack:25672 pid:5370 tgid:5370 ppid:5369 task_flags:0x400100 flags:0x00080000
[ 1112.629654] Call Trace:
[ 1112.633192]
[ 1112.636343] __schedule+0xcc3/0x19c0
[ 1112.641158] ? io_schedule_timeout+0x140/0x140
[ 1112.646934] ? find_held_lock+0x2b/0x80
[ 1112.652038] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.657923] schedule+0xd8/0x260
[ 1112.662340] schedule_preempt_disabled+0x18/0x30
[ 1112.668317] __mutex_lock+0x6af/0x2010
[ 1112.673320] ? threaded_store+0x132/0x220
[ 1112.678618] ? ww_mutex_lock+0x160/0x160
[ 1112.683817] ? selinux_inode_free_security+0x330/0x330
[ 1112.690386] ? cap_capable+0x1d6/0x3e0
[ 1112.695394] ? threaded_store+0x132/0x220
[ 1112.700690] threaded_store+0x132/0x220
[ 1112.705790] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.711083] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.716379] ? sysfs_kf_write+0x65/0x160
[ 1112.721579] ? sysfs_file_ops+0x140/0x140
[ 1112.726872] kernfs_fop_write_iter+0x335/0x510
[ 1112.732654] ? kernfs_ops+0xf0/0xf0
[ 1112.737365] new_sync_write+0x333/0x750
[ 1112.742457] ? __lock_acquire+0x508/0xc10
[ 1112.747752] ? new_sync_read+0x740/0x740
[ 1112.752950] ? lock_acquire.part.0+0xbc/0x260
[ 1112.758636] ? ksys_write+0x116/0x250
[ 1112.763543] vfs_write+0x657/0x9b0
[ 1112.768155] ksys_write+0x116/0x250
[ 1112.772867] ? __ia32_sys_read+0xc0/0xc0
[ 1112.778063] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.783650] ? rcu_is_watching+0x15/0xd0
[ 1112.788851] do_syscall_64+0xf3/0x650
[ 1112.793756] ? trace_hardirqs_off+0xd/0x30
[ 1112.799148] ? exc_page_fault+0xda/0xf0
[ 1112.804250] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.810716] RIP: 0033:0x7f930763864e
[ 1112.815522] RSP: 002b:00007ffe28f407d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.824818] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f930763864e
[ 1112.833624] RDX: 0000000000000001 RSI: 0000560d79773bf0 RDI: 0000000000000007
[ 1112.842429] RBP: 00007ffe28f407e0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.851234] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f93074b1390
[ 1112.860039] R13: 0000560d79773bf0 R14: 0000000000000007 R15: 00007f9307445120
[ 1112.868849]
[ 1112.872136] INFO: task python3:5370 is blocked on a mutex likely owned by task python3:3366.
[ 1112.882409] task:python3 state:D stack:25672 pid:3366 tgid:3366 ppid:3365 task_flags:0x400100 flags:0x00080802
[ 1112.895609] Call Trace:
[ 1112.899138]
[ 1112.902286] __schedule+0xcc3/0x19c0
[ 1112.907094] ? io_schedule_timeout+0x140/0x140
[ 1112.912879] ? find_held_lock+0x2b/0x80
[ 1112.917978] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.924157] schedule+0xd8/0x260
[ 1112.928576] schedule_timeout+0x13f/0x290
[ 1112.933873] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.940238] ? trace_contention_end+0xb3/0x1a0
[ 1112.946022] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.951711] ? ww_mutex_lock+0x160/0x160
[ 1112.956908] msleep+0x1b/0x30
[ 1112.961030] napi_set_threaded+0x3a5/0x440
[ 1112.966426] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.971921] netif_set_threaded+0x257/0x400
[ 1112.977412] threaded_store+0x1b8/0x220
[ 1112.982513] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.987807] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.993104] ? sysfs_kf_write+0x65/0x160
[ 1112.998305] ? sysfs_file_ops+0x140/0x140
[ 1113.003599] kernfs_fop_write_iter+0x335/0x510
[ 1113.009383] ? kernfs_ops+0xf0/0xf0
[ 1113.014092] new_sync_write+0x333/0x750
[ 1113.019193] ? __lock_acquire+0x508/0xc10
[ 1113.024488] ? new_sync_read+0x740/0x740
[ 1113.029686] ? lock_acquire.part.0+0xbc/0x260
[ 1113.035370] ? ksys_write+0x116/0x250
[ 1113.040285] vfs_write+0x657/0x9b0
[ 1113.044899] ksys_write+0x116/0x250
[ 1113.049609] ? __ia32_sys_read+0xc0/0xc0
[ 1113.054809] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.060389] ? rcu_is_watching+0x15/0xd0
[ 1113.065589] do_syscall_64+0xf3/0x650
[ 1113.070487] ? trace_hardirqs_off+0xd/0x30
[ 1113.075879] ? exc_page_fault+0xda/0xf0
[ 1113.080979] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.087445] RIP: 0033:0x7fab0446664e
[ 1113.092252] RSP: 002b:00007ffcee987e70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.101546] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fab0446664e
[ 1113.110354] RDX: 0000000000000001 RSI: 000055b701b3dbf0 RDI: 0000000000000007
[ 1113.119162] RBP: 00007ffcee987e80 R08: 0000000000000000 R09: 0000000000000000
[ 1113.127959] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fab042df390
[ 1113.136765] R13: 000055b701b3dbf0 R14: 0000000000000007 R15: 00007fab04275120
[ 1113.145575]
[ 1113.148837] \x0aShowing all locks held in the system:
[ 1113.157376] 1 lock held by khungtaskd/185:
[ 1113.162768] #0: ffffffff8ccfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.174248] 4 locks held by python3/3366:
[ 1113.179544] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.189434] #1: ff110001406d0880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.200394] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.211548] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.221643] 4 locks held by python3/5370:
[ 1113.226936] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.236824] #1: ff1100011a461a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.247782] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.258938] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.269026]
[ 1113.271492] =============================================\x0a
[ 1235.473598] INFO: task python3:5370 blocked for more than 368 seconds.
[ 1235.481747] Not tainted 7.1.0-rc7-gnws-g21ca060c075b #1
[ 1235.488997] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.498583] task:python3 state:D stack:25672 pid:5370 tgid:5370 ppid:5369 task_flags:0x400100 flags:0x00080002
[ 1235.511791] Call Trace:
[ 1235.515331]
[ 1235.518482] __schedule+0xcc3/0x19c0
[ 1235.523296] ? io_schedule_timeout+0x140/0x140
[ 1235.529082] ? find_held_lock+0x2b/0x80
[ 1235.534182] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.540067] schedule+0xd8/0x260
[ 1235.544483] schedule_preempt_disabled+0x18/0x30
[ 1235.550462] __mutex_lock+0x6af/0x2010
[ 1235.555465] ? threaded_store+0x132/0x220
[ 1235.560762] ? ww_mutex_lock+0x160/0x160
[ 1235.565962] ? selinux_inode_free_security+0x330/0x330
[ 1235.572531] ? cap_capable+0x1d6/0x3e0
[ 1235.577538] ? threaded_store+0x132/0x220
[ 1235.582836] threaded_store+0x132/0x220
[ 1235.587936] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.593234] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.598530] ? sysfs_kf_write+0x65/0x160
[ 1235.603728] ? sysfs_file_ops+0x140/0x140
[ 1235.609024] kernfs_fop_write_iter+0x335/0x510
[ 1235.614807] ? kernfs_ops+0xf0/0xf0
[ 1235.619515] new_sync_write+0x333/0x750
[ 1235.624619] ? __lock_acquire+0x508/0xc10
[ 1235.629905] ? new_sync_read+0x740/0x740
[ 1235.635105] ? lock_acquire.part.0+0xbc/0x260
[ 1235.640782] ? ksys_write+0x116/0x250
[ 1235.645692] vfs_write+0x657/0x9b0
[ 1235.650305] ksys_write+0x116/0x250
[ 1235.655013] ? __ia32_sys_read+0xc0/0xc0
[ 1235.660211] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.665797] ? rcu_is_watching+0x15/0xd0
[ 1235.670998] do_syscall_64+0xf3/0x650
[ 1235.675904] ? trace_hardirqs_off+0xd/0x30
[ 1235.681299] ? exc_page_fault+0xda/0xf0
[ 1235.686402] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.692868] RIP: 0033:0x7f930763864e
[ 1235.697675] RSP: 002b:00007ffe28f407d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.706970] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f930763864e
[ 1235.715776] RDX: 0000000000000001 RSI: 0000560d79773bf0 RDI: 0000000000000007
[ 1235.724582] RBP: 00007ffe28f407e0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.733388] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f93074b1390
[ 1235.742194] R13: 0000560d79773bf0 R14: 0000000000000007 R15: 00007f9307445120
[ 1235.751005]
[ 1235.754293] INFO: task python3:5370 is blocked on a mutex likely owned by task python3:3366.
[ 1235.764563] task:python3 state:D stack:25672 pid:3366 tgid:3366 ppid:3365 task_flags:0x400100 flags:0x00080802
[ 1235.777764] Call Trace:
[ 1235.781303]
[ 1235.784451] __schedule+0xcc3/0x19c0
[ 1235.789260] ? io_schedule_timeout+0x140/0x140
[ 1235.795042] ? find_held_lock+0x2b/0x80
[ 1235.800142] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.806321] schedule+0xd8/0x260
[ 1235.810740] schedule_timeout+0x13f/0x290
[ 1235.816034] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.822403] ? trace_contention_end+0xb3/0x1a0
[ 1235.828178] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.833866] ? ww_mutex_lock+0x160/0x160
[ 1235.839064] msleep+0x1b/0x30
[ 1235.843187] napi_set_threaded+0x3a5/0x440
[ 1235.848582] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.854076] netif_set_threaded+0x257/0x400
[ 1235.859569] threaded_store+0x1b8/0x220
[ 1235.864662] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.869958] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.875255] ? sysfs_kf_write+0x65/0x160
[ 1235.880452] ? sysfs_file_ops+0x140/0x140
[ 1235.885746] kernfs_fop_write_iter+0x335/0x510
[ 1235.891532] ? kernfs_ops+0xf0/0xf0
[ 1235.896235] new_sync_write+0x333/0x750
[ 1235.901336] ? __lock_acquire+0x508/0xc10
[ 1235.906631] ? new_sync_read+0x740/0x740
[ 1235.911830] ? lock_acquire.part.0+0xbc/0x260
[ 1235.917515] ? ksys_write+0x116/0x250
[ 1235.922426] vfs_write+0x657/0x9b0
[ 1235.927041] ksys_write+0x116/0x250
[ 1235.931742] ? __ia32_sys_read+0xc0/0xc0
[ 1235.936941] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.942530] ? rcu_is_watching+0x15/0xd0
[ 1235.947727] do_syscall_64+0xf3/0x650
[ 1235.952635] ? trace_hardirqs_off+0xd/0x30
[ 1235.958018] ? exc_page_fault+0xda/0xf0
[ 1235.963118] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.969585] RIP: 0033:0x7fab0446664e
[ 1235.974392] RSP: 002b:00007ffcee987e70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.983686] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fab0446664e
[ 1235.992494] RDX: 0000000000000001 RSI: 000055b701b3dbf0 RDI: 0000000000000007
[ 1236.001294] RBP: 00007ffcee987e80 R08: 0000000000000000 R09: 0000000000000000
[ 1236.010100] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fab042df390
[ 1236.018907] R13: 000055b701b3dbf0 R14: 0000000000000007 R15: 00007fab04275120
[ 1236.027719]
[ 1236.030984] \x0aShowing all locks held in the system:
[ 1236.039523] 1 lock held by khungtaskd/185:
[ 1236.044913] #0: ffffffff8ccfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.056392] 4 locks held by python3/3366:
[ 1236.061687] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.071574] #1: ff110001406d0880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.082534] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.093690] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.103785] 4 locks held by python3/5370:
[ 1236.109080] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.118965] #1: ff1100011a461a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.129925] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.141080] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.151168]
[ 1236.153634] =============================================\x0a
[ 1358.354762] INFO: task python3:5370 blocked for more than 491 seconds.
[ 1358.362912] Not tainted 7.1.0-rc7-gnws-g21ca060c075b #1
[ 1358.370162] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.379749] task:python3 state:D stack:25672 pid:5370 tgid:5370 ppid:5369 task_flags:0x400100 flags:0x00080002
[ 1358.392952] Call Trace:
[ 1358.396482]
[ 1358.399631] __schedule+0xcc3/0x19c0
[ 1358.404444] ? io_schedule_timeout+0x140/0x140
[ 1358.410232] ? find_held_lock+0x2b/0x80
[ 1358.415333] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.421215] schedule+0xd8/0x260
[ 1358.425623] schedule_preempt_disabled+0x18/0x30
[ 1358.431606] __mutex_lock+0x6af/0x2010
[ 1358.436611] ? threaded_store+0x132/0x220
[ 1358.441910] ? ww_mutex_lock+0x160/0x160
[ 1358.447101] ? selinux_inode_free_security+0x330/0x330
[ 1358.453671] ? cap_capable+0x1d6/0x3e0
[ 1358.458678] ? threaded_store+0x132/0x220
[ 1358.463973] threaded_store+0x132/0x220
[ 1358.469074] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.474371] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.479667] ? sysfs_kf_write+0x65/0x160
[ 1358.484865] ? sysfs_file_ops+0x140/0x140
[ 1358.490162] kernfs_fop_write_iter+0x335/0x510
[ 1358.495944] ? kernfs_ops+0xf0/0xf0
[ 1358.500656] new_sync_write+0x333/0x750
[ 1358.505758] ? __lock_acquire+0x508/0xc10
[ 1358.511054] ? new_sync_read+0x740/0x740
[ 1358.516256] ? lock_acquire.part.0+0xbc/0x260
[ 1358.521939] ? ksys_write+0x116/0x250
[ 1358.526848] vfs_write+0x657/0x9b0
[ 1358.531461] ksys_write+0x116/0x250
[ 1358.536169] ? __ia32_sys_read+0xc0/0xc0
[ 1358.541366] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.546953] ? rcu_is_watching+0x15/0xd0
[ 1358.552154] do_syscall_64+0xf3/0x650
[ 1358.557061] ? trace_hardirqs_off+0xd/0x30
[ 1358.562456] ? exc_page_fault+0xda/0xf0
[ 1358.567559] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.574025] RIP: 0033:0x7f930763864e
[ 1358.578831] RSP: 002b:00007ffe28f407d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.588127] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f930763864e
[ 1358.596935] RDX: 0000000000000001 RSI: 0000560d79773bf0 RDI: 0000000000000007
[ 1358.605740] RBP: 00007ffe28f407e0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.614548] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f93074b1390
[ 1358.623344] R13: 0000560d79773bf0 R14: 0000000000000007 R15: 00007f9307445120
[ 1358.632154]
[ 1358.635434] INFO: task python3:5370 is blocked on a mutex likely owned by task python3:3366.
[ 1358.645706] task:python3 state:D stack:25672 pid:3366 tgid:3366 ppid:3365 task_flags:0x400100 flags:0x00080802
[ 1358.658907] Call Trace:
[ 1358.662442]
[ 1358.665590] __schedule+0xcc3/0x19c0
[ 1358.670399] ? io_schedule_timeout+0x140/0x140
[ 1358.676185] ? find_held_lock+0x2b/0x80
[ 1358.681285] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.687462] schedule+0xd8/0x260
[ 1358.691881] schedule_timeout+0x13f/0x290
[ 1358.697174] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.703541] ? trace_contention_end+0xb3/0x1a0
[ 1358.709327] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.715016] ? ww_mutex_lock+0x160/0x160
[ 1358.720213] msleep+0x1b/0x30
[ 1358.724337] napi_set_threaded+0x3a5/0x440
[ 1358.729732] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.735228] netif_set_threaded+0x257/0x400
[ 1358.740719] threaded_store+0x1b8/0x220
[ 1358.745820] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.751115] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.756410] ? sysfs_kf_write+0x65/0x160
[ 1358.761610] ? sysfs_file_ops+0x140/0x140
[ 1358.766897] kernfs_fop_write_iter+0x335/0x510
[ 1358.772682] ? kernfs_ops+0xf0/0xf0
[ 1358.777391] new_sync_write+0x333/0x750
[ 1358.782492] ? __lock_acquire+0x508/0xc10
[ 1358.787788] ? new_sync_read+0x740/0x740
[ 1358.792988] ? lock_acquire.part.0+0xbc/0x260
[ 1358.798672] ? ksys_write+0x116/0x250
[ 1358.803581] vfs_write+0x657/0x9b0
[ 1358.808196] ksys_write+0x116/0x250
[ 1358.812905] ? __ia32_sys_read+0xc0/0xc0
[ 1358.818103] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.823691] ? rcu_is_watching+0x15/0xd0
[ 1358.828892] do_syscall_64+0xf3/0x650
[ 1358.833798] ? trace_hardirqs_off+0xd/0x30
[ 1358.839191] ? exc_page_fault+0xda/0xf0
[ 1358.844290] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.850755] RIP: 0033:0x7fab0446664e
[ 1358.855562] RSP: 002b:00007ffcee987e70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.864859] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fab0446664e
[ 1358.873659] RDX: 0000000000000001 RSI: 000055b701b3dbf0 RDI: 0000000000000007
[ 1358.882465] RBP: 00007ffcee987e80 R08: 0000000000000000 R09: 0000000000000000
[ 1358.891273] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fab042df390
[ 1358.900069] R13: 000055b701b3dbf0 R14: 0000000000000007 R15: 00007fab04275120
[ 1358.908883]
[ 1358.912156] \x0aShowing all locks held in the system:
[ 1358.920697] 1 lock held by khungtaskd/185:
[ 1358.926088] #0: ffffffff8ccfdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.937566] 4 locks held by python3/3366:
[ 1358.942861] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.952750] #1: ff110001406d0880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.963701] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.974856] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.984954] 4 locks held by python3/5370:
[ 1358.990248] #0: ff110001310093e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.000136] #1: ff1100011a461a80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.011096] #2: ff110001114b6b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.022251] #3: ff1100011c9a1e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.032340]
[ 1359.034806] =============================================\x0a