[ 763.247629] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 763.250122] nipa-hw-worker: [4/38] Retrying drivers/net:napi_threaded.py
[ 763.411302] kselftest: Running tests in drivers/net
[ 805.428578] kworker/u130:2 (406) used greatest stack depth: 21192 bytes left
[ 989.750260] INFO: task python3:5067 blocked for more than 122 seconds.
[ 989.758483] Not tainted 7.0.0-rc4-babn-g3ea2a328f638 #1
[ 989.765736] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.775326] task:python3 state:D stack:25312 pid:5067 tgid:5067 ppid:5066 task_flags:0x400100 flags:0x00080000
[ 989.788533] Call Trace:
[ 989.792073]
[ 989.795226] __schedule+0xad9/0x1490
[ 989.800043] ? io_schedule_timeout+0x130/0x130
[ 989.805827] ? find_held_lock+0x2b/0x80
[ 989.810933] ? __lock_release.isra.0+0x59/0x170
[ 989.816812] schedule+0xd4/0x260
[ 989.821231] schedule_preempt_disabled+0x18/0x30
[ 989.827210] __mutex_lock+0xd76/0x1f20
[ 989.832216] ? threaded_store+0x132/0x220
[ 989.837518] ? ww_mutex_lock+0x160/0x160
[ 989.842717] ? selinux_inode_free_security+0x310/0x310
[ 989.849290] ? cap_capable+0x1ed/0x3f0
[ 989.854297] ? sysfs_file_ops+0x140/0x140
[ 989.859598] ? threaded_store+0x132/0x220
[ 989.864897] threaded_store+0x132/0x220
[ 989.870001] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.875494] ? sysfs_file_ops+0x140/0x140
[ 989.880792] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.886091] ? sysfs_kf_write+0x65/0x160
[ 989.891292] kernfs_fop_write_iter+0x335/0x510
[ 989.897079] ? kernfs_ops+0xf0/0xf0
[ 989.901797] new_sync_write+0x308/0x6f0
[ 989.906902] ? new_sync_read+0x6f0/0x6f0
[ 989.912104] ? lock_acquire.part.0+0xbc/0x260
[ 989.917788] ? ksys_write+0xff/0x200
[ 989.922602] vfs_write+0x62a/0x980
[ 989.927220] ksys_write+0xff/0x200
[ 989.931836] ? __ia32_sys_read+0xc0/0xc0
[ 989.937034] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.942625] ? rcu_is_watching+0x15/0xd0
[ 989.947829] do_syscall_64+0xf3/0x5e0
[ 989.952736] ? exc_page_fault+0x9e/0xb0
[ 989.957840] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.964307] RIP: 0033:0x7f734f414c5e
[ 989.969116] RSP: 002b:00007ffceadd26c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.978417] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f734f414c5e
[ 989.987225] RDX: 0000000000000001 RSI: 00005598489c0110 RDI: 0000000000000007
[ 989.996033] RBP: 00007ffceadd26d0 R08: 0000000000000000 R09: 0000000000000000
[ 990.004832] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f734f2afe80
[ 990.013634] R13: 00005598489c0110 R14: 0000000000000007 R15: 00007f734fb15b98
[ 990.022447]
[ 990.025713] INFO: task python3:5067 is blocked on a mutex likely owned by task python3:3132.
[ 990.035985] task:python3 state:D stack:25312 pid:3132 tgid:3132 ppid:3131 task_flags:0x400100 flags:0x00080802
[ 990.049187] Call Trace:
[ 990.052724]
[ 990.055875] __schedule+0xad9/0x1490
[ 990.060685] ? io_schedule_timeout+0x130/0x130
[ 990.066467] ? find_held_lock+0x2b/0x80
[ 990.071570] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.077751] schedule+0xd4/0x260
[ 990.082167] schedule_timeout+0x103/0x260
[ 990.087465] ? rcu_is_watching+0x15/0xd0
[ 990.092663] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.099033] ? hrtimers_cpu_dying+0x250/0x250
[ 990.104715] ? ww_mutex_lock+0x160/0x160
[ 990.109913] msleep+0x1b/0x30
[ 990.114038] napi_set_threaded+0x376/0x410
[ 990.119436] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.124932] netif_set_threaded+0x254/0x400
[ 990.130425] ? sysfs_file_ops+0x140/0x140
[ 990.135722] threaded_store+0x1bb/0x220
[ 990.140831] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.146322] ? sysfs_file_ops+0x140/0x140
[ 990.151618] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.156916] ? sysfs_kf_write+0x65/0x160
[ 990.162116] kernfs_fop_write_iter+0x335/0x510
[ 990.167895] ? kernfs_ops+0xf0/0xf0
[ 990.172606] new_sync_write+0x308/0x6f0
[ 990.177710] ? new_sync_read+0x6f0/0x6f0
[ 990.182910] ? lock_acquire.part.0+0xbc/0x260
[ 990.188594] ? ksys_write+0xff/0x200
[ 990.193408] vfs_write+0x62a/0x980
[ 990.198024] ksys_write+0xff/0x200
[ 990.202640] ? __ia32_sys_read+0xc0/0xc0
[ 990.207839] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.213421] ? rcu_is_watching+0x15/0xd0
[ 990.218612] do_syscall_64+0xf3/0x5e0
[ 990.223510] ? irq_exit_rcu+0x39/0xf0
[ 990.228417] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.234886] RIP: 0033:0x7f52063fdc5e
[ 990.239692] RSP: 002b:00007fff7e5f0e20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.248988] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f52063fdc5e
[ 990.257785] RDX: 0000000000000001 RSI: 000055f89596c110 RDI: 0000000000000007
[ 990.266594] RBP: 00007fff7e5f0e30 R08: 0000000000000000 R09: 0000000000000000
[ 990.275401] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5206298e80
[ 990.284209] R13: 000055f89596c110 R14: 0000000000000007 R15: 00007f5206226480
[ 990.293021]
[ 990.296281] \x0aShowing all locks held in the system:
[ 990.304820] 1 lock held by khungtaskd/184:
[ 990.310214] #0: ffffffff924ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.321687] 3 locks held by systemd-journal/809:
[ 990.327664] #0: ff1100180de37220 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 990.337550] #1: ffffffff926c8ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.350367] #2: ff110001a1a9c020 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 990.361828] 4 locks held by python3/3132:
[ 990.367124] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.376919] #1: ff1100013dc1e888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.387883] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.399040] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.409134] 4 locks held by python3/5067:
[ 990.414430] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.424221] #1: ff110001a0980888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.435184] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.446340] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.456431]
[ 990.458890] =============================================\x0a
[ 1112.632393] INFO: task python3:5067 blocked for more than 245 seconds.
[ 1112.640541] Not tainted 7.0.0-rc4-babn-g3ea2a328f638 #1
[ 1112.647790] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.657382] task:python3 state:D stack:25312 pid:5067 tgid:5067 ppid:5066 task_flags:0x400100 flags:0x00080000
[ 1112.670588] Call Trace:
[ 1112.674129]
[ 1112.677282] __schedule+0xad9/0x1490
[ 1112.682101] ? io_schedule_timeout+0x130/0x130
[ 1112.687887] ? find_held_lock+0x2b/0x80
[ 1112.692996] ? __lock_release.isra.0+0x59/0x170
[ 1112.698881] schedule+0xd4/0x260
[ 1112.703299] schedule_preempt_disabled+0x18/0x30
[ 1112.709277] __mutex_lock+0xd76/0x1f20
[ 1112.714282] ? threaded_store+0x132/0x220
[ 1112.719581] ? ww_mutex_lock+0x160/0x160
[ 1112.724782] ? selinux_inode_free_security+0x310/0x310
[ 1112.731352] ? cap_capable+0x1ed/0x3f0
[ 1112.736360] ? sysfs_file_ops+0x140/0x140
[ 1112.741660] ? threaded_store+0x132/0x220
[ 1112.746947] threaded_store+0x132/0x220
[ 1112.752050] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.757544] ? sysfs_file_ops+0x140/0x140
[ 1112.762845] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.768142] ? sysfs_kf_write+0x65/0x160
[ 1112.773335] kernfs_fop_write_iter+0x335/0x510
[ 1112.779122] ? kernfs_ops+0xf0/0xf0
[ 1112.783835] new_sync_write+0x308/0x6f0
[ 1112.788939] ? new_sync_read+0x6f0/0x6f0
[ 1112.794131] ? lock_acquire.part.0+0xbc/0x260
[ 1112.799818] ? ksys_write+0xff/0x200
[ 1112.804630] vfs_write+0x62a/0x980
[ 1112.809245] ksys_write+0xff/0x200
[ 1112.813851] ? __ia32_sys_read+0xc0/0xc0
[ 1112.819040] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.824628] ? rcu_is_watching+0x15/0xd0
[ 1112.829830] do_syscall_64+0xf3/0x5e0
[ 1112.834736] ? exc_page_fault+0x9e/0xb0
[ 1112.839841] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.846307] RIP: 0033:0x7f734f414c5e
[ 1112.851120] RSP: 002b:00007ffceadd26c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.860419] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f734f414c5e
[ 1112.869230] RDX: 0000000000000001 RSI: 00005598489c0110 RDI: 0000000000000007
[ 1112.878037] RBP: 00007ffceadd26d0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.886847] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f734f2afe80
[ 1112.895657] R13: 00005598489c0110 R14: 0000000000000007 R15: 00007f734fb15b98
[ 1112.904473]
[ 1112.907745] INFO: task python3:5067 is blocked on a mutex likely owned by task python3:3132.
[ 1112.918018] task:python3 state:D stack:25312 pid:3132 tgid:3132 ppid:3131 task_flags:0x400100 flags:0x00080802
[ 1112.931222] Call Trace:
[ 1112.934755]
[ 1112.937907] __schedule+0xad9/0x1490
[ 1112.942720] ? io_schedule_timeout+0x130/0x130
[ 1112.948501] ? find_held_lock+0x2b/0x80
[ 1112.953603] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.959774] schedule+0xd4/0x260
[ 1112.964194] schedule_timeout+0x103/0x260
[ 1112.969492] ? rcu_is_watching+0x15/0xd0
[ 1112.974690] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.981062] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.986754] ? ww_mutex_lock+0x160/0x160
[ 1112.991952] msleep+0x1b/0x30
[ 1112.996079] napi_set_threaded+0x376/0x410
[ 1113.001464] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.006960] netif_set_threaded+0x254/0x400
[ 1113.012456] ? sysfs_file_ops+0x140/0x140
[ 1113.017752] threaded_store+0x1bb/0x220
[ 1113.022856] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.028347] ? sysfs_file_ops+0x140/0x140
[ 1113.033645] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.038943] ? sysfs_kf_write+0x65/0x160
[ 1113.044147] kernfs_fop_write_iter+0x335/0x510
[ 1113.049932] ? kernfs_ops+0xf0/0xf0
[ 1113.054637] new_sync_write+0x308/0x6f0
[ 1113.059740] ? new_sync_read+0x6f0/0x6f0
[ 1113.064934] ? lock_acquire.part.0+0xbc/0x260
[ 1113.070620] ? ksys_write+0xff/0x200
[ 1113.075432] vfs_write+0x62a/0x980
[ 1113.080047] ksys_write+0xff/0x200
[ 1113.084660] ? __ia32_sys_read+0xc0/0xc0
[ 1113.089859] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.095450] ? rcu_is_watching+0x15/0xd0
[ 1113.100650] do_syscall_64+0xf3/0x5e0
[ 1113.105554] ? irq_exit_rcu+0x39/0xf0
[ 1113.110464] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.116932] RIP: 0033:0x7f52063fdc5e
[ 1113.121742] RSP: 002b:00007fff7e5f0e20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.131040] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f52063fdc5e
[ 1113.139848] RDX: 0000000000000001 RSI: 000055f89596c110 RDI: 0000000000000007
[ 1113.148658] RBP: 00007fff7e5f0e30 R08: 0000000000000000 R09: 0000000000000000
[ 1113.157465] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5206298e80
[ 1113.166267] R13: 000055f89596c110 R14: 0000000000000007 R15: 00007f5206226480
[ 1113.175083]
[ 1113.178336] \x0aShowing all locks held in the system:
[ 1113.186875] 1 lock held by khungtaskd/184:
[ 1113.192268] #0: ffffffff924ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.203752] 4 locks held by python3/3132:
[ 1113.209048] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.218834] #1: ff1100013dc1e888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.229796] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.240955] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.251048] 4 locks held by python3/5067:
[ 1113.256342] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.266131] #1: ff110001a0980888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.277092] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.288243] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.298338]
[ 1113.300806] =============================================\x0a
[ 1235.513544] INFO: task python3:5067 blocked for more than 368 seconds.
[ 1235.521695] Not tainted 7.0.0-rc4-babn-g3ea2a328f638 #1
[ 1235.528944] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.538524] task:python3 state:D stack:25312 pid:5067 tgid:5067 ppid:5066 task_flags:0x400100 flags:0x00080002
[ 1235.551732] Call Trace:
[ 1235.555272]
[ 1235.558424] __schedule+0xad9/0x1490
[ 1235.563241] ? io_schedule_timeout+0x130/0x130
[ 1235.569025] ? find_held_lock+0x2b/0x80
[ 1235.574133] ? __lock_release.isra.0+0x59/0x170
[ 1235.580015] schedule+0xd4/0x260
[ 1235.584431] schedule_preempt_disabled+0x18/0x30
[ 1235.590410] __mutex_lock+0xd76/0x1f20
[ 1235.595413] ? threaded_store+0x132/0x220
[ 1235.600701] ? ww_mutex_lock+0x160/0x160
[ 1235.605901] ? selinux_inode_free_security+0x310/0x310
[ 1235.612469] ? cap_capable+0x1ed/0x3f0
[ 1235.617476] ? sysfs_file_ops+0x140/0x140
[ 1235.622778] ? threaded_store+0x132/0x220
[ 1235.628074] threaded_store+0x132/0x220
[ 1235.633176] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.638666] ? sysfs_file_ops+0x140/0x140
[ 1235.643962] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.649257] ? sysfs_kf_write+0x65/0x160
[ 1235.654457] kernfs_fop_write_iter+0x335/0x510
[ 1235.660234] ? kernfs_ops+0xf0/0xf0
[ 1235.664933] new_sync_write+0x308/0x6f0
[ 1235.670036] ? new_sync_read+0x6f0/0x6f0
[ 1235.675235] ? lock_acquire.part.0+0xbc/0x260
[ 1235.680911] ? ksys_write+0xff/0x200
[ 1235.685723] vfs_write+0x62a/0x980
[ 1235.690338] ksys_write+0xff/0x200
[ 1235.694953] ? __ia32_sys_read+0xc0/0xc0
[ 1235.700149] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.705737] ? rcu_is_watching+0x15/0xd0
[ 1235.710937] do_syscall_64+0xf3/0x5e0
[ 1235.715842] ? exc_page_fault+0x9e/0xb0
[ 1235.720944] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.727410] RIP: 0033:0x7f734f414c5e
[ 1235.732217] RSP: 002b:00007ffceadd26c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.741515] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f734f414c5e
[ 1235.750321] RDX: 0000000000000001 RSI: 00005598489c0110 RDI: 0000000000000007
[ 1235.759130] RBP: 00007ffceadd26d0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.767939] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f734f2afe80
[ 1235.776744] R13: 00005598489c0110 R14: 0000000000000007 R15: 00007f734fb15b98
[ 1235.785554]
[ 1235.788833] INFO: task python3:5067 is blocked on a mutex likely owned by task python3:3132.
[ 1235.799105] task:python3 state:D stack:25312 pid:3132 tgid:3132 ppid:3131 task_flags:0x400100 flags:0x00080802
[ 1235.812305] Call Trace:
[ 1235.815843]
[ 1235.818992] __schedule+0xad9/0x1490
[ 1235.823803] ? io_schedule_timeout+0x130/0x130
[ 1235.829584] ? find_held_lock+0x2b/0x80
[ 1235.834684] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.840863] schedule+0xd4/0x260
[ 1235.845278] schedule_timeout+0x103/0x260
[ 1235.850575] ? rcu_is_watching+0x15/0xd0
[ 1235.855772] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.862139] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.867827] ? ww_mutex_lock+0x160/0x160
[ 1235.873028] msleep+0x1b/0x30
[ 1235.877155] napi_set_threaded+0x376/0x410
[ 1235.882548] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.888043] netif_set_threaded+0x254/0x400
[ 1235.893533] ? sysfs_file_ops+0x140/0x140
[ 1235.898828] threaded_store+0x1bb/0x220
[ 1235.903928] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.909419] ? sysfs_file_ops+0x140/0x140
[ 1235.914712] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.920008] ? sysfs_kf_write+0x65/0x160
[ 1235.925207] kernfs_fop_write_iter+0x335/0x510
[ 1235.930992] ? kernfs_ops+0xf0/0xf0
[ 1235.935702] new_sync_write+0x308/0x6f0
[ 1235.940805] ? new_sync_read+0x6f0/0x6f0
[ 1235.945993] ? lock_acquire.part.0+0xbc/0x260
[ 1235.951676] ? ksys_write+0xff/0x200
[ 1235.956488] vfs_write+0x62a/0x980
[ 1235.961100] ksys_write+0xff/0x200
[ 1235.965714] ? __ia32_sys_read+0xc0/0xc0
[ 1235.970912] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.976501] ? rcu_is_watching+0x15/0xd0
[ 1235.981701] do_syscall_64+0xf3/0x5e0
[ 1235.986606] ? irq_exit_rcu+0x39/0xf0
[ 1235.991511] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.997978] RIP: 0033:0x7f52063fdc5e
[ 1236.002783] RSP: 002b:00007fff7e5f0e20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.012077] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f52063fdc5e
[ 1236.020883] RDX: 0000000000000001 RSI: 000055f89596c110 RDI: 0000000000000007
[ 1236.029689] RBP: 00007fff7e5f0e30 R08: 0000000000000000 R09: 0000000000000000
[ 1236.038496] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5206298e80
[ 1236.047301] R13: 000055f89596c110 R14: 0000000000000007 R15: 00007f5206226480
[ 1236.056113]
[ 1236.059384] \x0aShowing all locks held in the system:
[ 1236.067920] 1 lock held by khungtaskd/184:
[ 1236.073313] #0: ffffffff924ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.084793] 4 locks held by python3/3132:
[ 1236.090086] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.099874] #1: ff1100013dc1e888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.110835] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.121991] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.132086] 4 locks held by python3/5067:
[ 1236.137386] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.147173] #1: ff110001a0980888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.158134] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.169292] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.179388]
[ 1236.181852] =============================================\x0a
[ 1358.394694] INFO: task python3:5067 blocked for more than 491 seconds.
[ 1358.402840] Not tainted 7.0.0-rc4-babn-g3ea2a328f638 #1
[ 1358.410089] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.419675] task:python3 state:D stack:25312 pid:5067 tgid:5067 ppid:5066 task_flags:0x400100 flags:0x00080002
[ 1358.432883] Call Trace:
[ 1358.436422]
[ 1358.439575] __schedule+0xad9/0x1490
[ 1358.444388] ? io_schedule_timeout+0x130/0x130
[ 1358.450173] ? find_held_lock+0x2b/0x80
[ 1358.455279] ? __lock_release.isra.0+0x59/0x170
[ 1358.461162] schedule+0xd4/0x260
[ 1358.465577] schedule_preempt_disabled+0x18/0x30
[ 1358.471553] __mutex_lock+0xd76/0x1f20
[ 1358.476556] ? threaded_store+0x132/0x220
[ 1358.481854] ? ww_mutex_lock+0x160/0x160
[ 1358.487049] ? selinux_inode_free_security+0x310/0x310
[ 1358.493618] ? cap_capable+0x1ed/0x3f0
[ 1358.498627] ? sysfs_file_ops+0x140/0x140
[ 1358.503927] ? threaded_store+0x132/0x220
[ 1358.509221] threaded_store+0x132/0x220
[ 1358.514323] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.519812] ? sysfs_file_ops+0x140/0x140
[ 1358.525107] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.530402] ? sysfs_kf_write+0x65/0x160
[ 1358.535602] kernfs_fop_write_iter+0x335/0x510
[ 1358.541379] ? kernfs_ops+0xf0/0xf0
[ 1358.546087] new_sync_write+0x308/0x6f0
[ 1358.551189] ? new_sync_read+0x6f0/0x6f0
[ 1358.556390] ? lock_acquire.part.0+0xbc/0x260
[ 1358.562076] ? ksys_write+0xff/0x200
[ 1358.566886] vfs_write+0x62a/0x980
[ 1358.571491] ksys_write+0xff/0x200
[ 1358.576104] ? __ia32_sys_read+0xc0/0xc0
[ 1358.581300] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.586887] ? rcu_is_watching+0x15/0xd0
[ 1358.592088] do_syscall_64+0xf3/0x5e0
[ 1358.596993] ? exc_page_fault+0x9e/0xb0
[ 1358.602093] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.608558] RIP: 0033:0x7f734f414c5e
[ 1358.613366] RSP: 002b:00007ffceadd26c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.622663] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f734f414c5e
[ 1358.631463] RDX: 0000000000000001 RSI: 00005598489c0110 RDI: 0000000000000007
[ 1358.640268] RBP: 00007ffceadd26d0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.649073] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f734f2afe80
[ 1358.657878] R13: 00005598489c0110 R14: 0000000000000007 R15: 00007f734fb15b98
[ 1358.666690]
[ 1358.669953] INFO: task python3:5067 is blocked on a mutex likely owned by task python3:3132.
[ 1358.680225] task:python3 state:D stack:25312 pid:3132 tgid:3132 ppid:3131 task_flags:0x400100 flags:0x00080802
[ 1358.693429] Call Trace:
[ 1358.696968]
[ 1358.700117] __schedule+0xad9/0x1490
[ 1358.704924] ? io_schedule_timeout+0x130/0x130
[ 1358.710705] ? find_held_lock+0x2b/0x80
[ 1358.715806] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.721978] schedule+0xd4/0x260
[ 1358.726394] schedule_timeout+0x103/0x260
[ 1358.731691] ? rcu_is_watching+0x15/0xd0
[ 1358.736890] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.743258] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.748947] ? ww_mutex_lock+0x160/0x160
[ 1358.754145] msleep+0x1b/0x30
[ 1358.758268] napi_set_threaded+0x376/0x410
[ 1358.763662] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.769157] netif_set_threaded+0x254/0x400
[ 1358.774650] ? sysfs_file_ops+0x140/0x140
[ 1358.779937] threaded_store+0x1bb/0x220
[ 1358.785038] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.790530] ? sysfs_file_ops+0x140/0x140
[ 1358.795825] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.801123] ? sysfs_kf_write+0x65/0x160
[ 1358.806313] kernfs_fop_write_iter+0x335/0x510
[ 1358.812094] ? kernfs_ops+0xf0/0xf0
[ 1358.816805] new_sync_write+0x308/0x6f0
[ 1358.821909] ? new_sync_read+0x6f0/0x6f0
[ 1358.827108] ? lock_acquire.part.0+0xbc/0x260
[ 1358.832791] ? ksys_write+0xff/0x200
[ 1358.837605] vfs_write+0x62a/0x980
[ 1358.842216] ksys_write+0xff/0x200
[ 1358.846831] ? __ia32_sys_read+0xc0/0xc0
[ 1358.852021] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.857611] ? rcu_is_watching+0x15/0xd0
[ 1358.862808] do_syscall_64+0xf3/0x5e0
[ 1358.867714] ? irq_exit_rcu+0x39/0xf0
[ 1358.872621] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.879089] RIP: 0033:0x7f52063fdc5e
[ 1358.883895] RSP: 002b:00007fff7e5f0e20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.893190] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f52063fdc5e
[ 1358.901996] RDX: 0000000000000001 RSI: 000055f89596c110 RDI: 0000000000000007
[ 1358.910803] RBP: 00007fff7e5f0e30 R08: 0000000000000000 R09: 0000000000000000
[ 1358.919610] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5206298e80
[ 1358.928418] R13: 000055f89596c110 R14: 0000000000000007 R15: 00007f5206226480
[ 1358.937228]
[ 1358.940494] \x0aShowing all locks held in the system:
[ 1358.949024] 1 lock held by khungtaskd/184:
[ 1358.954416] #0: ffffffff924ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.965897] 4 locks held by python3/3132:
[ 1358.971194] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.980983] #1: ff1100013dc1e888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.991944] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.003099] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.013190] 4 locks held by python3/5067:
[ 1359.018481] #0: ff1100012ef443f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.028270] #1: ff110001a0980888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.039227] #2: ff11000121fa50f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.050383] #3: ff11000122421cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.060476]
[ 1359.062941] =============================================\x0a