[ 762.543793] nipa-hw-worker: drivers/net:napi_threaded.py: timed out [ 762.546342] nipa-hw-worker: [4/39] Retrying drivers/net:napi_threaded.py [ 762.706055] kselftest: Running tests in drivers/net [ 989.738699] INFO: task python3:5017 blocked for more than 122 seconds. [ 989.746936] Not tainted 7.0.0-rc4-gkwk-ge3c926123b3c #1 [ 989.754187] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.763774] task:python3 state:D stack:25312 pid:5017 tgid:5017 ppid:5016 task_flags:0x400100 flags:0x00080000 [ 989.776981] Call Trace: [ 989.780517] [ 989.783667] __schedule+0xad9/0x1490 [ 989.788484] ? io_schedule_timeout+0x130/0x130 [ 989.794270] ? find_held_lock+0x2b/0x80 [ 989.799377] ? __lock_release.isra.0+0x59/0x170 [ 989.805261] schedule+0xd4/0x260 [ 989.809669] schedule_preempt_disabled+0x18/0x30 [ 989.815649] __mutex_lock+0xd76/0x1f20 [ 989.820644] ? threaded_store+0x132/0x220 [ 989.825943] ? ww_mutex_lock+0x160/0x160 [ 989.831139] ? selinux_inode_free_security+0x310/0x310 [ 989.837708] ? cap_capable+0x1ed/0x3f0 [ 989.842714] ? sysfs_file_ops+0x140/0x140 [ 989.848012] ? threaded_store+0x132/0x220 [ 989.853305] threaded_store+0x132/0x220 [ 989.858410] ? tx_maxrate_store+0x4e0/0x4e0 [ 989.863902] ? sysfs_file_ops+0x140/0x140 [ 989.869189] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.874486] ? sysfs_kf_write+0x65/0x160 [ 989.879684] kernfs_fop_write_iter+0x335/0x510 [ 989.885466] ? kernfs_ops+0xf0/0xf0 [ 989.890175] new_sync_write+0x308/0x6f0 [ 989.895279] ? new_sync_read+0x6f0/0x6f0 [ 989.900468] ? lock_acquire.part.0+0xbc/0x260 [ 989.906154] ? ksys_write+0xff/0x200 [ 989.910966] vfs_write+0x62a/0x980 [ 989.915580] ksys_write+0xff/0x200 [ 989.920192] ? __ia32_sys_read+0xc0/0xc0 [ 989.925388] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.930978] ? rcu_is_watching+0x15/0xd0 [ 989.936171] do_syscall_64+0xf3/0x5e0 [ 989.941078] ? irq_exit_rcu+0x39/0xf0 [ 989.945974] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.952442] RIP: 0033:0x7f430c033c5e [ 989.957250] RSP: 002b:00007ffdbb8ce6b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.966552] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f430c033c5e [ 989.975359] RDX: 0000000000000001 RSI: 0000560c3f9ed110 RDI: 0000000000000007 [ 989.984166] RBP: 00007ffdbb8ce6c0 R08: 0000000000000000 R09: 0000000000000000 [ 989.992971] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f430becee80 [ 990.001778] R13: 0000560c3f9ed110 R14: 0000000000000007 R15: 00007f430c734b98 [ 990.010589] [ 990.013856] INFO: task python3:5017 is blocked on a mutex likely owned by task python3:3092. [ 990.024118] task:python3 state:D stack:25312 pid:3092 tgid:3092 ppid:3091 task_flags:0x400100 flags:0x00080802 [ 990.037318] Call Trace: [ 990.040848] [ 990.043996] __schedule+0xad9/0x1490 [ 990.048807] ? io_schedule_timeout+0x130/0x130 [ 990.054588] ? find_held_lock+0x2b/0x80 [ 990.059691] ? __try_to_del_timer_sync+0x154/0x1f0 [ 990.065869] schedule+0xd4/0x260 [ 990.070286] schedule_timeout+0x103/0x260 [ 990.075579] ? rcu_is_watching+0x15/0xd0 [ 990.080777] ? hrtimer_nanosleep_restart+0x180/0x180 [ 990.087145] ? hrtimers_cpu_dying+0x250/0x250 [ 990.092834] ? ww_mutex_lock+0x160/0x160 [ 990.098033] msleep+0x1b/0x30 [ 990.102157] napi_set_threaded+0x376/0x410 [ 990.107550] ? napi_resume_irqs+0x1a0/0x1a0 [ 990.113044] netif_set_threaded+0x254/0x400 [ 990.118539] ? sysfs_file_ops+0x140/0x140 [ 990.123836] threaded_store+0x1bb/0x220 [ 990.128935] ? tx_maxrate_store+0x4e0/0x4e0 [ 990.134423] ? sysfs_file_ops+0x140/0x140 [ 990.139717] ? sysfs_file_kobj+0xbd/0x1e0 [ 990.145011] ? sysfs_kf_write+0x65/0x160 [ 990.150209] kernfs_fop_write_iter+0x335/0x510 [ 990.155993] ? kernfs_ops+0xf0/0xf0 [ 990.160705] new_sync_write+0x308/0x6f0 [ 990.165806] ? new_sync_read+0x6f0/0x6f0 [ 990.171004] ? lock_acquire.part.0+0xbc/0x260 [ 990.176689] ? ksys_write+0xff/0x200 [ 990.181499] vfs_write+0x62a/0x980 [ 990.186107] ksys_write+0xff/0x200 [ 990.190721] ? __ia32_sys_read+0xc0/0xc0 [ 990.195911] ? kernfs_fop_llseek+0x160/0x1f0 [ 990.201501] ? rcu_is_watching+0x15/0xd0 [ 990.206699] do_syscall_64+0xf3/0x5e0 [ 990.211602] ? irq_exit_rcu+0x39/0xf0 [ 990.216509] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 990.222974] RIP: 0033:0x7f091b86fc5e [ 990.227781] RSP: 002b:00007fff88ac0d60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 990.237077] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f091b86fc5e [ 990.245885] RDX: 0000000000000001 RSI: 000055b532603110 RDI: 0000000000000007 [ 990.254692] RBP: 00007fff88ac0d70 R08: 0000000000000000 R09: 0000000000000000 [ 990.263500] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f091b70ae80 [ 990.272307] R13: 000055b532603110 R14: 0000000000000007 R15: 00007f091b69a480 [ 990.281121] [ 990.284381] \x0aShowing all locks held in the system: [ 990.292916] 1 lock held by khungtaskd/185: [ 990.298306] #0: ffffffff98cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 990.309785] 4 locks held by python3/3092: [ 990.315079] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 990.324869] #1: ff1100013d1ae888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 990.335830] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.346987] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.357080] 4 locks held by python3/5017: [ 990.362373] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 990.372160] #1: ff1100019ffda088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 990.383120] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.394276] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.404365] [ 990.406828] =============================================\x0a [ 1112.619856] INFO: task python3:5017 blocked for more than 245 seconds. [ 1112.628005] Not tainted 7.0.0-rc4-gkwk-ge3c926123b3c #1 [ 1112.635257] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.644848] task:python3 state:D stack:25312 pid:5017 tgid:5017 ppid:5016 task_flags:0x400100 flags:0x00080000 [ 1112.658054] Call Trace: [ 1112.661595] [ 1112.664744] __schedule+0xad9/0x1490 [ 1112.669560] ? io_schedule_timeout+0x130/0x130 [ 1112.675346] ? find_held_lock+0x2b/0x80 [ 1112.680455] ? __lock_release.isra.0+0x59/0x170 [ 1112.686341] schedule+0xd4/0x260 [ 1112.690760] schedule_preempt_disabled+0x18/0x30 [ 1112.696730] __mutex_lock+0xd76/0x1f20 [ 1112.701737] ? threaded_store+0x132/0x220 [ 1112.707036] ? ww_mutex_lock+0x160/0x160 [ 1112.712234] ? selinux_inode_free_security+0x310/0x310 [ 1112.718797] ? cap_capable+0x1ed/0x3f0 [ 1112.723807] ? sysfs_file_ops+0x140/0x140 [ 1112.729105] ? threaded_store+0x132/0x220 [ 1112.734405] threaded_store+0x132/0x220 [ 1112.739508] ? tx_maxrate_store+0x4e0/0x4e0 [ 1112.745001] ? sysfs_file_ops+0x140/0x140 [ 1112.750291] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.755588] ? sysfs_kf_write+0x65/0x160 [ 1112.760789] kernfs_fop_write_iter+0x335/0x510 [ 1112.766577] ? kernfs_ops+0xf0/0xf0 [ 1112.771292] new_sync_write+0x308/0x6f0 [ 1112.776398] ? new_sync_read+0x6f0/0x6f0 [ 1112.781599] ? lock_acquire.part.0+0xbc/0x260 [ 1112.787287] ? ksys_write+0xff/0x200 [ 1112.792101] vfs_write+0x62a/0x980 [ 1112.796717] ksys_write+0xff/0x200 [ 1112.801331] ? __ia32_sys_read+0xc0/0xc0 [ 1112.806523] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.812104] ? rcu_is_watching+0x15/0xd0 [ 1112.817304] do_syscall_64+0xf3/0x5e0 [ 1112.822203] ? irq_exit_rcu+0x39/0xf0 [ 1112.827111] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.833662] RIP: 0033:0x7f430c033c5e [ 1112.838472] RSP: 002b:00007ffdbb8ce6b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.847763] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f430c033c5e [ 1112.856574] RDX: 0000000000000001 RSI: 0000560c3f9ed110 RDI: 0000000000000007 [ 1112.865383] RBP: 00007ffdbb8ce6c0 R08: 0000000000000000 R09: 0000000000000000 [ 1112.874193] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f430becee80 [ 1112.883002] R13: 0000560c3f9ed110 R14: 0000000000000007 R15: 00007f430c734b98 [ 1112.891819] [ 1112.895090] INFO: task python3:5017 is blocked on a mutex likely owned by task python3:3092. [ 1112.905362] task:python3 state:D stack:25312 pid:3092 tgid:3092 ppid:3091 task_flags:0x400100 flags:0x00080802 [ 1112.918563] Call Trace: [ 1112.922101] [ 1112.925252] __schedule+0xad9/0x1490 [ 1112.930064] ? io_schedule_timeout+0x130/0x130 [ 1112.935850] ? find_held_lock+0x2b/0x80 [ 1112.940955] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1112.947137] schedule+0xd4/0x260 [ 1112.951559] schedule_timeout+0x103/0x260 [ 1112.956858] ? rcu_is_watching+0x15/0xd0 [ 1112.962130] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1112.968499] ? hrtimers_cpu_dying+0x250/0x250 [ 1112.974183] ? ww_mutex_lock+0x160/0x160 [ 1112.979384] msleep+0x1b/0x30 [ 1112.983582] napi_set_threaded+0x376/0x410 [ 1112.988980] ? napi_resume_irqs+0x1a0/0x1a0 [ 1112.994477] netif_set_threaded+0x254/0x400 [ 1112.999975] ? sysfs_file_ops+0x140/0x140 [ 1113.005269] threaded_store+0x1bb/0x220 [ 1113.010374] ? tx_maxrate_store+0x4e0/0x4e0 [ 1113.015869] ? sysfs_file_ops+0x140/0x140 [ 1113.021167] ? sysfs_file_kobj+0xbd/0x1e0 [ 1113.026457] ? sysfs_kf_write+0x65/0x160 [ 1113.031660] kernfs_fop_write_iter+0x335/0x510 [ 1113.037448] ? kernfs_ops+0xf0/0xf0 [ 1113.042160] new_sync_write+0x308/0x6f0 [ 1113.047265] ? new_sync_read+0x6f0/0x6f0 [ 1113.052468] ? lock_acquire.part.0+0xbc/0x260 [ 1113.058157] ? ksys_write+0xff/0x200 [ 1113.062969] vfs_write+0x62a/0x980 [ 1113.067587] ksys_write+0xff/0x200 [ 1113.072201] ? __ia32_sys_read+0xc0/0xc0 [ 1113.077404] ? kernfs_fop_llseek+0x160/0x1f0 [ 1113.082993] ? rcu_is_watching+0x15/0xd0 [ 1113.088193] do_syscall_64+0xf3/0x5e0 [ 1113.093098] ? irq_exit_rcu+0x39/0xf0 [ 1113.098005] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1113.104464] RIP: 0033:0x7f091b86fc5e [ 1113.109265] RSP: 002b:00007fff88ac0d60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1113.118555] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f091b86fc5e [ 1113.127362] RDX: 0000000000000001 RSI: 000055b532603110 RDI: 0000000000000007 [ 1113.136171] RBP: 00007fff88ac0d70 R08: 0000000000000000 R09: 0000000000000000 [ 1113.144978] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f091b70ae80 [ 1113.153788] R13: 000055b532603110 R14: 0000000000000007 R15: 00007f091b69a480 [ 1113.162604] [ 1113.165870] \x0aShowing all locks held in the system: [ 1113.174407] 1 lock held by khungtaskd/185: [ 1113.179799] #0: ffffffff98cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1113.191351] 4 locks held by python3/3092: [ 1113.196644] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.206437] #1: ff1100013d1ae888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.217402] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.228562] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.238655] 4 locks held by python3/5017: [ 1113.244015] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.253807] #1: ff1100019ffda088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.264772] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.275934] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.286026] [ 1113.288495] =============================================\x0a [ 1235.500037] INFO: task python3:5017 blocked for more than 368 seconds. [ 1235.508183] Not tainted 7.0.0-rc4-gkwk-ge3c926123b3c #1 [ 1235.515433] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.525025] task:python3 state:D stack:25312 pid:5017 tgid:5017 ppid:5016 task_flags:0x400100 flags:0x00080002 [ 1235.538232] Call Trace: [ 1235.541772] [ 1235.544922] __schedule+0xad9/0x1490 [ 1235.549738] ? io_schedule_timeout+0x130/0x130 [ 1235.555524] ? find_held_lock+0x2b/0x80 [ 1235.560632] ? __lock_release.isra.0+0x59/0x170 [ 1235.566517] schedule+0xd4/0x260 [ 1235.570936] schedule_preempt_disabled+0x18/0x30 [ 1235.576915] __mutex_lock+0xd76/0x1f20 [ 1235.581921] ? threaded_store+0x132/0x220 [ 1235.587221] ? ww_mutex_lock+0x160/0x160 [ 1235.592408] ? selinux_inode_free_security+0x310/0x310 [ 1235.598977] ? cap_capable+0x1ed/0x3f0 [ 1235.603986] ? sysfs_file_ops+0x140/0x140 [ 1235.609286] ? threaded_store+0x132/0x220 [ 1235.614581] threaded_store+0x132/0x220 [ 1235.619685] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.625178] ? sysfs_file_ops+0x140/0x140 [ 1235.630476] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.635762] ? sysfs_kf_write+0x65/0x160 [ 1235.640964] kernfs_fop_write_iter+0x335/0x510 [ 1235.646750] ? kernfs_ops+0xf0/0xf0 [ 1235.651464] new_sync_write+0x308/0x6f0 [ 1235.656567] ? new_sync_read+0x6f0/0x6f0 [ 1235.661769] ? lock_acquire.part.0+0xbc/0x260 [ 1235.667455] ? ksys_write+0xff/0x200 [ 1235.672268] vfs_write+0x62a/0x980 [ 1235.676884] ksys_write+0xff/0x200 [ 1235.681498] ? __ia32_sys_read+0xc0/0xc0 [ 1235.686694] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.692286] ? rcu_is_watching+0x15/0xd0 [ 1235.697491] do_syscall_64+0xf3/0x5e0 [ 1235.702395] ? irq_exit_rcu+0x39/0xf0 [ 1235.707304] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.713765] RIP: 0033:0x7f430c033c5e [ 1235.718576] RSP: 002b:00007ffdbb8ce6b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.727876] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f430c033c5e [ 1235.736685] RDX: 0000000000000001 RSI: 0000560c3f9ed110 RDI: 0000000000000007 [ 1235.745493] RBP: 00007ffdbb8ce6c0 R08: 0000000000000000 R09: 0000000000000000 [ 1235.754302] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f430becee80 [ 1235.763111] R13: 0000560c3f9ed110 R14: 0000000000000007 R15: 00007f430c734b98 [ 1235.771927] [ 1235.775194] INFO: task python3:5017 is blocked on a mutex likely owned by task python3:3092. [ 1235.785468] task:python3 state:D stack:25312 pid:3092 tgid:3092 ppid:3091 task_flags:0x400100 flags:0x00080802 [ 1235.798670] Call Trace: [ 1235.802208] [ 1235.805360] __schedule+0xad9/0x1490 [ 1235.810172] ? io_schedule_timeout+0x130/0x130 [ 1235.815957] ? find_held_lock+0x2b/0x80 [ 1235.821052] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1235.827232] schedule+0xd4/0x260 [ 1235.831651] schedule_timeout+0x103/0x260 [ 1235.836949] ? rcu_is_watching+0x15/0xd0 [ 1235.842142] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1235.848512] ? hrtimers_cpu_dying+0x250/0x250 [ 1235.854204] ? ww_mutex_lock+0x160/0x160 [ 1235.859403] msleep+0x1b/0x30 [ 1235.863520] napi_set_threaded+0x376/0x410 [ 1235.868915] ? napi_resume_irqs+0x1a0/0x1a0 [ 1235.874408] netif_set_threaded+0x254/0x400 [ 1235.879903] ? sysfs_file_ops+0x140/0x140 [ 1235.885197] threaded_store+0x1bb/0x220 [ 1235.890300] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.895790] ? sysfs_file_ops+0x140/0x140 [ 1235.901087] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.906384] ? sysfs_kf_write+0x65/0x160 [ 1235.911588] kernfs_fop_write_iter+0x335/0x510 [ 1235.917373] ? kernfs_ops+0xf0/0xf0 [ 1235.922086] new_sync_write+0x308/0x6f0 [ 1235.927187] ? new_sync_read+0x6f0/0x6f0 [ 1235.932389] ? lock_acquire.part.0+0xbc/0x260 [ 1235.938075] ? ksys_write+0xff/0x200 [ 1235.942889] vfs_write+0x62a/0x980 [ 1235.947504] ksys_write+0xff/0x200 [ 1235.952117] ? __ia32_sys_read+0xc0/0xc0 [ 1235.957318] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.962910] ? rcu_is_watching+0x15/0xd0 [ 1235.968113] do_syscall_64+0xf3/0x5e0 [ 1235.973018] ? irq_exit_rcu+0x39/0xf0 [ 1235.977925] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.984393] RIP: 0033:0x7f091b86fc5e [ 1235.989201] RSP: 002b:00007fff88ac0d60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.998497] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f091b86fc5e [ 1236.007305] RDX: 0000000000000001 RSI: 000055b532603110 RDI: 0000000000000007 [ 1236.016114] RBP: 00007fff88ac0d70 R08: 0000000000000000 R09: 0000000000000000 [ 1236.024923] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f091b70ae80 [ 1236.033731] R13: 000055b532603110 R14: 0000000000000007 R15: 00007f091b69a480 [ 1236.042543] [ 1236.045814] \x0aShowing all locks held in the system: [ 1236.054350] 1 lock held by khungtaskd/185: [ 1236.059746] #0: ffffffff98cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1236.071211] 3 locks held by systemd-journal/809: [ 1236.077190] #0: ff1100180dd37220 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490 [ 1236.087080] #1: ffffffff98ec8ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120 [ 1236.099897] #2: ff11000187534bd0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280 [ 1236.111356] 4 locks held by python3/3092: [ 1236.116653] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1236.126444] #1: ff1100013d1ae888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1236.137398] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.148559] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.158652] 4 locks held by python3/5017: [ 1236.163947] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1236.173738] #1: ff1100019ffda088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1236.184689] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.195845] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.205936] [ 1236.208392] =============================================\x0a [ 1358.381222] INFO: task python3:5017 blocked for more than 491 seconds. [ 1358.389368] Not tainted 7.0.0-rc4-gkwk-ge3c926123b3c #1 [ 1358.396616] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.406206] task:python3 state:D stack:25312 pid:5017 tgid:5017 ppid:5016 task_flags:0x400100 flags:0x00080002 [ 1358.419403] Call Trace: [ 1358.422942] [ 1358.426089] __schedule+0xad9/0x1490 [ 1358.430893] ? io_schedule_timeout+0x130/0x130 [ 1358.436679] ? find_held_lock+0x2b/0x80 [ 1358.441785] ? __lock_release.isra.0+0x59/0x170 [ 1358.447669] schedule+0xd4/0x260 [ 1358.452085] schedule_preempt_disabled+0x18/0x30 [ 1358.458066] __mutex_lock+0xd76/0x1f20 [ 1358.463071] ? threaded_store+0x132/0x220 [ 1358.468369] ? ww_mutex_lock+0x160/0x160 [ 1358.473567] ? selinux_inode_free_security+0x310/0x310 [ 1358.480135] ? cap_capable+0x1ed/0x3f0 [ 1358.485142] ? sysfs_file_ops+0x140/0x140 [ 1358.490437] ? threaded_store+0x132/0x220 [ 1358.495732] threaded_store+0x132/0x220 [ 1358.500835] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.506327] ? sysfs_file_ops+0x140/0x140 [ 1358.511624] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.516910] ? sysfs_kf_write+0x65/0x160 [ 1358.522111] kernfs_fop_write_iter+0x335/0x510 [ 1358.527897] ? kernfs_ops+0xf0/0xf0 [ 1358.532609] new_sync_write+0x308/0x6f0 [ 1358.537711] ? new_sync_read+0x6f0/0x6f0 [ 1358.542913] ? lock_acquire.part.0+0xbc/0x260 [ 1358.548598] ? ksys_write+0xff/0x200 [ 1358.553409] vfs_write+0x62a/0x980 [ 1358.558014] ksys_write+0xff/0x200 [ 1358.562627] ? __ia32_sys_read+0xc0/0xc0 [ 1358.567822] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.573409] ? rcu_is_watching+0x15/0xd0 [ 1358.578609] do_syscall_64+0xf3/0x5e0 [ 1358.583512] ? irq_exit_rcu+0x39/0xf0 [ 1358.588420] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.594886] RIP: 0033:0x7f430c033c5e [ 1358.599695] RSP: 002b:00007ffdbb8ce6b0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.608993] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f430c033c5e [ 1358.617789] RDX: 0000000000000001 RSI: 0000560c3f9ed110 RDI: 0000000000000007 [ 1358.626596] RBP: 00007ffdbb8ce6c0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.635401] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f430becee80 [ 1358.644206] R13: 0000560c3f9ed110 R14: 0000000000000007 R15: 00007f430c734b98 [ 1358.653022] [ 1358.656291] INFO: task python3:5017 is blocked on a mutex likely owned by task python3:3092. [ 1358.666560] task:python3 state:D stack:25312 pid:3092 tgid:3092 ppid:3091 task_flags:0x400100 flags:0x00080802 [ 1358.679763] Call Trace: [ 1358.683302] [ 1358.686449] __schedule+0xad9/0x1490 [ 1358.691257] ? io_schedule_timeout+0x130/0x130 [ 1358.697038] ? find_held_lock+0x2b/0x80 [ 1358.702139] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1358.708319] schedule+0xd4/0x260 [ 1358.712738] schedule_timeout+0x103/0x260 [ 1358.718026] ? rcu_is_watching+0x15/0xd0 [ 1358.723224] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1358.729592] ? hrtimers_cpu_dying+0x250/0x250 [ 1358.735280] ? ww_mutex_lock+0x160/0x160 [ 1358.740467] msleep+0x1b/0x30 [ 1358.744590] napi_set_threaded+0x376/0x410 [ 1358.749985] ? napi_resume_irqs+0x1a0/0x1a0 [ 1358.755479] netif_set_threaded+0x254/0x400 [ 1358.760971] ? sysfs_file_ops+0x140/0x140 [ 1358.766266] threaded_store+0x1bb/0x220 [ 1358.771366] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.776858] ? sysfs_file_ops+0x140/0x140 [ 1358.782151] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.787448] ? sysfs_kf_write+0x65/0x160 [ 1358.792645] kernfs_fop_write_iter+0x335/0x510 [ 1358.798428] ? kernfs_ops+0xf0/0xf0 [ 1358.803138] new_sync_write+0x308/0x6f0 [ 1358.808241] ? new_sync_read+0x6f0/0x6f0 [ 1358.813440] ? lock_acquire.part.0+0xbc/0x260 [ 1358.819124] ? ksys_write+0xff/0x200 [ 1358.823937] vfs_write+0x62a/0x980 [ 1358.828552] ksys_write+0xff/0x200 [ 1358.833163] ? __ia32_sys_read+0xc0/0xc0 [ 1358.838362] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.843950] ? rcu_is_watching+0x15/0xd0 [ 1358.849151] do_syscall_64+0xf3/0x5e0 [ 1358.854057] ? irq_exit_rcu+0x39/0xf0 [ 1358.858964] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.865430] RIP: 0033:0x7f091b86fc5e [ 1358.870236] RSP: 002b:00007fff88ac0d60 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.879530] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f091b86fc5e [ 1358.888337] RDX: 0000000000000001 RSI: 000055b532603110 RDI: 0000000000000007 [ 1358.897143] RBP: 00007fff88ac0d70 R08: 0000000000000000 R09: 0000000000000000 [ 1358.905950] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f091b70ae80 [ 1358.914755] R13: 000055b532603110 R14: 0000000000000007 R15: 00007f091b69a480 [ 1358.923567] [ 1358.926836] \x0aShowing all locks held in the system: [ 1358.935370] 1 lock held by khungtaskd/185: [ 1358.940764] #0: ffffffff98cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1358.952242] 4 locks held by python3/3092: [ 1358.957535] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1358.967325] #1: ff1100013d1ae888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1358.978286] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1358.989441] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1358.999532] 4 locks held by python3/5017: [ 1359.004818] #0: ff1100012ec813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1359.014608] #1: ff1100019ffda088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1359.025562] #2: ff11000121d0b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1359.036721] #3: ff11000121dc9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1359.046814] [ 1359.049280] =============================================\x0a