[ 758.635689] nipa-hw-worker: drivers/net:napi_threaded.py: timed out [ 758.638236] nipa-hw-worker: [4/38] Retrying drivers/net:napi_threaded.py [ 758.774977] kselftest: Running tests in drivers/net [ 989.701943] INFO: task python3:4985 blocked for more than 122 seconds. [ 989.710160] Not tainted 7.0.0-rc4-tivk-g9b3c8f4f703c #1 [ 989.717410] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.726999] task:python3 state:D stack:25312 pid:4985 tgid:4985 ppid:4984 task_flags:0x400100 flags:0x00080000 [ 989.740201] Call Trace: [ 989.743740] [ 989.746891] __schedule+0xad9/0x1490 [ 989.751705] ? io_schedule_timeout+0x130/0x130 [ 989.757482] ? find_held_lock+0x2b/0x80 [ 989.762589] ? __lock_release.isra.0+0x59/0x170 [ 989.768474] schedule+0xd4/0x260 [ 989.772891] schedule_preempt_disabled+0x18/0x30 [ 989.778867] __mutex_lock+0xd76/0x1f20 [ 989.783872] ? threaded_store+0x132/0x220 [ 989.789171] ? ww_mutex_lock+0x160/0x160 [ 989.794367] ? selinux_inode_free_security+0x310/0x310 [ 989.800937] ? cap_capable+0x1ed/0x3f0 [ 989.805946] ? sysfs_file_ops+0x140/0x140 [ 989.811244] ? threaded_store+0x132/0x220 [ 989.816539] threaded_store+0x132/0x220 [ 989.821641] ? tx_maxrate_store+0x4e0/0x4e0 [ 989.827125] ? sysfs_file_ops+0x140/0x140 [ 989.832419] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.837716] ? sysfs_kf_write+0x65/0x160 [ 989.842914] kernfs_fop_write_iter+0x335/0x510 [ 989.848696] ? kernfs_ops+0xf0/0xf0 [ 989.853405] new_sync_write+0x308/0x6f0 [ 989.858508] ? new_sync_read+0x6f0/0x6f0 [ 989.863708] ? lock_acquire.part.0+0xbc/0x260 [ 989.869395] ? ksys_write+0xff/0x200 [ 989.874207] vfs_write+0x62a/0x980 [ 989.878822] ksys_write+0xff/0x200 [ 989.883436] ? __ia32_sys_read+0xc0/0xc0 [ 989.888632] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.894222] ? rcu_is_watching+0x15/0xd0 [ 989.899422] do_syscall_64+0xf3/0x5e0 [ 989.904328] ? exc_page_fault+0x9e/0xb0 [ 989.909431] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.915898] RIP: 0033:0x7f7363e4dc5e [ 989.920697] RSP: 002b:00007fffafde34f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.930003] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7363e4dc5e [ 989.938810] RDX: 0000000000000001 RSI: 0000559eb2865110 RDI: 0000000000000007 [ 989.947609] RBP: 00007fffafde3500 R08: 0000000000000000 R09: 0000000000000000 [ 989.956414] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7363ce8e80 [ 989.965221] R13: 0000559eb2865110 R14: 0000000000000007 R15: 00007f736454eb98 [ 989.974032] [ 989.977303] INFO: task python3:4985 is blocked on a mutex likely owned by task python3:3059. [ 989.987576] task:python3 state:D stack:25312 pid:3059 tgid:3059 ppid:3058 task_flags:0x400100 flags:0x00080802 [ 990.000779] Call Trace: [ 990.004318] [ 990.007468] __schedule+0xad9/0x1490 [ 990.012275] ? io_schedule_timeout+0x130/0x130 [ 990.018057] ? find_held_lock+0x2b/0x80 [ 990.023158] ? __try_to_del_timer_sync+0x154/0x1f0 [ 990.029338] schedule+0xd4/0x260 [ 990.033753] schedule_timeout+0x103/0x260 [ 990.039047] ? rcu_is_watching+0x15/0xd0 [ 990.044245] ? hrtimer_nanosleep_restart+0x180/0x180 [ 990.050614] ? hrtimers_cpu_dying+0x250/0x250 [ 990.056304] ? ww_mutex_lock+0x160/0x160 [ 990.061500] msleep+0x1b/0x30 [ 990.065627] napi_set_threaded+0x376/0x410 [ 990.071022] ? napi_resume_irqs+0x1a0/0x1a0 [ 990.076517] netif_set_threaded+0x254/0x400 [ 990.082008] ? sysfs_file_ops+0x140/0x140 [ 990.087303] threaded_store+0x1bb/0x220 [ 990.092404] ? tx_maxrate_store+0x4e0/0x4e0 [ 990.097897] ? sysfs_file_ops+0x140/0x140 [ 990.103193] ? sysfs_file_kobj+0xbd/0x1e0 [ 990.108489] ? sysfs_kf_write+0x65/0x160 [ 990.113687] kernfs_fop_write_iter+0x335/0x510 [ 990.119471] ? kernfs_ops+0xf0/0xf0 [ 990.124183] new_sync_write+0x308/0x6f0 [ 990.129286] ? new_sync_read+0x6f0/0x6f0 [ 990.134485] ? lock_acquire.part.0+0xbc/0x260 [ 990.140168] ? ksys_write+0xff/0x200 [ 990.144981] vfs_write+0x62a/0x980 [ 990.149592] ksys_write+0xff/0x200 [ 990.154206] ? __ia32_sys_read+0xc0/0xc0 [ 990.159402] ? kernfs_fop_llseek+0x160/0x1f0 [ 990.164990] ? rcu_is_watching+0x15/0xd0 [ 990.170187] do_syscall_64+0xf3/0x5e0 [ 990.175090] ? exc_page_fault+0x9e/0xb0 [ 990.180191] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 990.186658] RIP: 0033:0x7fa1525c5c5e [ 990.191462] RSP: 002b:00007fff8b3d5540 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 990.200756] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1525c5c5e [ 990.209564] RDX: 0000000000000001 RSI: 000055ab232f3110 RDI: 0000000000000007 [ 990.218370] RBP: 00007fff8b3d5550 R08: 0000000000000000 R09: 0000000000000000 [ 990.227179] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa152460e80 [ 990.235986] R13: 000055ab232f3110 R14: 0000000000000007 R15: 00007fa1523ee480 [ 990.244795] [ 990.248051] \x0aShowing all locks held in the system: [ 990.256589] 1 lock held by khungtaskd/186: [ 990.261982] #0: ffffffff87ece020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 990.273462] 4 locks held by python3/3059: [ 990.278756] #0: ff1100012d7813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 990.288544] #1: ff1100013bfbfa88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 990.299506] #2: ff110001202dd0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.310662] #3: ff11000120349cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.320756] 4 locks held by python3/4985: [ 990.326048] #0: ff1100012d7813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 990.335837] #1: ff11000205cf7a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 990.346800] #2: ff110001202dd0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.357958] #3: ff11000120349cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.368047] [ 990.370511] =============================================\x0a [ 1112.584155] INFO: task python3:4985 blocked for more than 245 seconds. [ 1112.592299] Not tainted 7.0.0-rc4-tivk-g9b3c8f4f703c #1 [ 1112.599552] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.609145] task:python3 state:D stack:25312 pid:4985 tgid:4985 ppid:4984 task_flags:0x400100 flags:0x00080000 [ 1112.622352] Call Trace: [ 1112.625891] [ 1112.629044] __schedule+0xad9/0x1490 [ 1112.633863] ? io_schedule_timeout+0x130/0x130 [ 1112.639649] ? find_held_lock+0x2b/0x80 [ 1112.644756] ? __lock_release.isra.0+0x59/0x170 [ 1112.650644] schedule+0xd4/0x260 [ 1112.655063] schedule_preempt_disabled+0x18/0x30 [ 1112.661042] __mutex_lock+0xd76/0x1f20 [ 1112.666046] ? threaded_store+0x132/0x220 [ 1112.671345] ? ww_mutex_lock+0x160/0x160 [ 1112.676542] ? selinux_inode_free_security+0x310/0x310 [ 1112.683109] ? cap_capable+0x1ed/0x3f0 [ 1112.688115] ? sysfs_file_ops+0x140/0x140 [ 1112.693412] ? threaded_store+0x132/0x220 [ 1112.698709] threaded_store+0x132/0x220 [ 1112.703814] ? tx_maxrate_store+0x4e0/0x4e0 [ 1112.709308] ? sysfs_file_ops+0x140/0x140 [ 1112.714605] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.719902] ? sysfs_kf_write+0x65/0x160 [ 1112.725092] kernfs_fop_write_iter+0x335/0x510 [ 1112.730875] ? kernfs_ops+0xf0/0xf0 [ 1112.735584] new_sync_write+0x308/0x6f0 [ 1112.740686] ? new_sync_read+0x6f0/0x6f0 [ 1112.745886] ? lock_acquire.part.0+0xbc/0x260 [ 1112.751573] ? ksys_write+0xff/0x200 [ 1112.756377] vfs_write+0x62a/0x980 [ 1112.760998] ksys_write+0xff/0x200 [ 1112.765611] ? __ia32_sys_read+0xc0/0xc0 [ 1112.770811] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.776393] ? rcu_is_watching+0x15/0xd0 [ 1112.781598] do_syscall_64+0xf3/0x5e0 [ 1112.786507] ? exc_page_fault+0x9e/0xb0 [ 1112.791609] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.798078] RIP: 0033:0x7f7363e4dc5e [ 1112.802890] RSP: 002b:00007fffafde34f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.812188] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7363e4dc5e [ 1112.820997] RDX: 0000000000000001 RSI: 0000559eb2865110 RDI: 0000000000000007 [ 1112.829805] RBP: 00007fffafde3500 R08: 0000000000000000 R09: 0000000000000000 [ 1112.838614] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7363ce8e80 [ 1112.847423] R13: 0000559eb2865110 R14: 0000000000000007 R15: 00007f736454eb98 [ 1112.856236] [ 1112.859514] INFO: task python3:4985 is blocked on a mutex likely owned by task python3:3059. [ 1112.869788] task:python3 state:D stack:25312 pid:3059 tgid:3059 ppid:3058 task_flags:0x400100 flags:0x00080802 [ 1112.882995] Call Trace: [ 1112.886536] [ 1112.889687] __schedule+0xad9/0x1490 [ 1112.894490] ? io_schedule_timeout+0x130/0x130 [ 1112.900272] ? find_held_lock+0x2b/0x80 [ 1112.905378] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1112.905386] schedule+0xd4/0x260 [ 1112.905390] schedule_timeout+0x103/0x260 [ 1112.905393] ? rcu_is_watching+0x15/0xd0 [ 1112.905396] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1112.905400] ? hrtimers_cpu_dying+0x250/0x250 [ 1112.938569] ? ww_mutex_lock+0x160/0x160 [ 1112.943770] msleep+0x1b/0x30 [ 1112.947895] napi_set_threaded+0x376/0x410 [ 1112.953290] ? napi_resume_irqs+0x1a0/0x1a0 [ 1112.958785] netif_set_threaded+0x254/0x400 [ 1112.964281] ? sysfs_file_ops+0x140/0x140 [ 1112.969578] threaded_store+0x1bb/0x220 [ 1112.974682] ? tx_maxrate_store+0x4e0/0x4e0 [ 1112.980174] ? sysfs_file_ops+0x140/0x140 [ 1112.985469] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.990766] ? sysfs_kf_write+0x65/0x160 [ 1112.995965] kernfs_fop_write_iter+0x335/0x510 [ 1113.001749] ? kernfs_ops+0xf0/0xf0 [ 1113.006460] new_sync_write+0x308/0x6f0 [ 1113.011564] ? new_sync_read+0x6f0/0x6f0 [ 1113.016763] ? lock_acquire.part.0+0xbc/0x260 [ 1113.022452] ? ksys_write+0xff/0x200 [ 1113.027264] vfs_write+0x62a/0x980 [ 1113.031882] ksys_write+0xff/0x200 [ 1113.036495] ? __ia32_sys_read+0xc0/0xc0 [ 1113.041693] ? kernfs_fop_llseek+0x160/0x1f0 [ 1113.047283] ? rcu_is_watching+0x15/0xd0 [ 1113.052485] do_syscall_64+0xf3/0x5e0 [ 1113.057390] ? exc_page_fault+0x9e/0xb0 [ 1113.062493] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1113.068963] RIP: 0033:0x7fa1525c5c5e [ 1113.073772] RSP: 002b:00007fff8b3d5540 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1113.083061] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1525c5c5e [ 1113.091859] RDX: 0000000000000001 RSI: 000055ab232f3110 RDI: 0000000000000007 [ 1113.100659] RBP: 00007fff8b3d5550 R08: 0000000000000000 R09: 0000000000000000 [ 1113.109466] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa152460e80 [ 1113.118265] R13: 000055ab232f3110 R14: 0000000000000007 R15: 00007fa1523ee480 [ 1113.127074] [ 1113.130339] \x0aShowing all locks held in the system: [ 1113.138872] 1 lock held by khungtaskd/186: [ 1113.144267] #0: ffffffff87ece020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1113.155747] 4 locks held by python3/3059: [ 1113.161041] #0: ff1100012d7813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.170835] #1: ff1100013bfbfa88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.181801] #2: ff110001202dd0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.192959] #3: ff11000120349cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.203054] 4 locks held by python3/4985: [ 1113.208350] #0: ff1100012d7813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.218143] #1: ff11000205cf7a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.229106] #2: ff110001202dd0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.240265] #3: ff11000120349cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.250356] 5 locks held by (time-dir)/6061: [ 1113.255945] #0: ff1100012d8a13f8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.265735] #1: ff110001f4966e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.276698] #2: ffffffff87f36e48 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xd2/0x3b0 [ 1113.287369] #3: ffffffff87d68e90 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e7/0x640 [ 1113.299014] #4: ffffffff87f36b50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f3/0x640 [ 1113.311441] [ 1113.313910] =============================================\x0a [ 1235.465379] INFO: task python3:4985 blocked for more than 368 seconds. [ 1235.473527] Not tainted 7.0.0-rc4-tivk-g9b3c8f4f703c #1 [ 1235.480779] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.490368] task:python3 state:D stack:25312 pid:4985 tgid:4985 ppid:4984 task_flags:0x400100 flags:0x00080002 [ 1235.503566] Call Trace: [ 1235.507105] [ 1235.510259] __schedule+0xad9/0x1490 [ 1235.515077] ? io_schedule_timeout+0x130/0x130 [ 1235.520862] ? find_held_lock+0x2b/0x80 [ 1235.525968] ? __lock_release.isra.0+0x59/0x170 [ 1235.531853] schedule+0xd4/0x260 [ 1235.536272] schedule_preempt_disabled+0x18/0x30 [ 1235.542245] __mutex_lock+0xd76/0x1f20 [ 1235.547253] ? threaded_store+0x132/0x220 [ 1235.552555] ? ww_mutex_lock+0x160/0x160 [ 1235.557755] ? selinux_inode_free_security+0x310/0x310 [ 1235.564324] ? cap_capable+0x1ed/0x3f0 [ 1235.569332] ? sysfs_file_ops+0x140/0x140 [ 1235.574631] ? threaded_store+0x132/0x220 [ 1235.579929] threaded_store+0x132/0x220 [ 1235.585031] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.590529] ? sysfs_file_ops+0x140/0x140 [ 1235.595828] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.601126] ? sysfs_kf_write+0x65/0x160 [ 1235.606326] kernfs_fop_write_iter+0x335/0x510 [ 1235.612102] ? kernfs_ops+0xf0/0xf0 [ 1235.616815] new_sync_write+0x308/0x6f0 [ 1235.621919] ? new_sync_read+0x6f0/0x6f0 [ 1235.627120] ? lock_acquire.part.0+0xbc/0x260 [ 1235.632806] ? ksys_write+0xff/0x200 [ 1235.637617] vfs_write+0x62a/0x980 [ 1235.642233] ksys_write+0xff/0x200 [ 1235.646849] ? __ia32_sys_read+0xc0/0xc0 [ 1235.652050] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.657642] ? rcu_is_watching+0x15/0xd0 [ 1235.662846] do_syscall_64+0xf3/0x5e0 [ 1235.667754] ? exc_page_fault+0x9e/0xb0 [ 1235.672847] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.679309] RIP: 0033:0x7f7363e4dc5e [ 1235.684119] RSP: 002b:00007fffafde34f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.693419] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7363e4dc5e [ 1235.702233] RDX: 0000000000000001 RSI: 0000559eb2865110 RDI: 0000000000000007 [ 1235.711042] RBP: 00007fffafde3500 R08: 0000000000000000 R09: 0000000000000000 [ 1235.719850] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7363ce8e80 [ 1235.728659] R13: 0000559eb2865110 R14: 0000000000000007 R15: 00007f736454eb98 [ 1235.737475] [ 1235.740739] INFO: task python3:4985 is blocked on a mutex likely owned by task python3:3059. [ 1235.751014] task:python3 state:D stack:25312 pid:3059 tgid:3059 ppid:3058 task_flags:0x400100 flags:0x00080802 [ 1235.764215] Call Trace: [ 1235.767756] [ 1235.770906] __schedule+0xad9/0x1490 [ 1235.775716] ? io_schedule_timeout+0x130/0x130 [ 1235.781491] ? find_held_lock+0x2b/0x80 [ 1235.786595] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1235.792774] schedule+0xd4/0x260 [ 1235.797193] schedule_timeout+0x103/0x260 [ 1235.802484] ? rcu_is_watching+0x15/0xd0 [ 1235.807685] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1235.814055] ? hrtimers_cpu_dying+0x250/0x250 [ 1235.819745] ? ww_mutex_lock+0x160/0x160 [ 1235.824946] msleep+0x1b/0x30 [ 1235.829073] napi_set_threaded+0x376/0x410 [ 1235.834470] ? napi_resume_irqs+0x1a0/0x1a0 [ 1235.839969] netif_set_threaded+0x254/0x400 [ 1235.845462] ? sysfs_file_ops+0x140/0x140 [ 1235.850757] threaded_store+0x1bb/0x220 [ 1235.855860] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.861354] ? sysfs_file_ops+0x140/0x140 [ 1235.866650] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.871949] ? sysfs_kf_write+0x65/0x160 [ 1235.877149] kernfs_fop_write_iter+0x335/0x510 [ 1235.882934] ? kernfs_ops+0xf0/0xf0 [ 1235.887647] new_sync_write+0x308/0x6f0 [ 1235.892752] ? new_sync_read+0x6f0/0x6f0 [ 1235.897953] ? lock_acquire.part.0+0xbc/0x260 [ 1235.903637] ? ksys_write+0xff/0x200 [ 1235.908451] vfs_write+0x62a/0x980 [ 1235.913068] ksys_write+0xff/0x200 [ 1235.917681] ? __ia32_sys_read+0xc0/0xc0 [ 1235.922881] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.928463] ? rcu_is_watching+0x15/0xd0 [ 1235.933666] do_syscall_64+0xf3/0x5e0 [ 1235.938573] ? exc_page_fault+0x9e/0xb0 [ 1235.943675] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.950142] RIP: 0033:0x7fa1525c5c5e [ 1235.954944] RSP: 002b:00007fff8b3d5540 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.964241] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fa1525c5c5e [ 1235.973052] RDX: 0000000000000001 RSI: 000055ab232f3110 RDI: 0000000000000007 [ 1235.981861] RBP: 00007fff8b3d5550 R08: 0000000000000000 R09: 0000000000000000 [ 1235.990672] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa152460e80 [ 1235.999479] R13: 000055ab232f3110 R14: 0000000000000007 R15: 00007fa1523ee480 [ 1236.008285] [ 1236.011551] \x0aShowing all locks held in the system: [ 1236.020083] 1 lock held by khungtaskd/186: [ 1236.025478] #0: ffffffff87ece020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1236.036951] 4 locks held by python3/3059: [ 1236.042245] #0: ff1100012d7813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1236.052038] #1: ff1100013bfbfa88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1236.063003] #2: ff110001202dd0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.074160] #3: ff11000120349cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.084256] 4 locks held by python3/4985: [ 1236.089550] #0: ff1100012d7813f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1236.099340] #1: ff11000205cf7a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1236.110303] #2: ff110001202dd0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.121463] #3: ff11000120349cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.131556] [ 1236.134021] =============================================\x0a [ 1358.346622] INFO: task python3:4985 blocked for more than 491 seconds. [ 1358.354767] Not tainted 7.0.0-rc4-tivk-g9b3c8f4f703c #1 [ 1358.362006] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.371594] task:python3 state:D stack:25312 pid:4985 tgid:4985 ppid:4984 task_flags:0x400100 flags:0x00080002 [ 1358.384800] Call Trace: [ 1358.388338] [ 1358.391486] __schedule+0xad9/0x1490 [ 1358.396300] ? io_schedule_timeout+0x130/0x130 [ 1358.402084] ? find_held_lock+0x2b/0x80 [ 1358.407187] ? __lock_release.isra.0+0x59/0x170 [ 1358.413072] schedule+0xd4/0x260 [ 1358.417490] schedule_preempt_disabled+0x18/0x30 [ 1358.423470] __mutex_lock+0xd76/0x1f20 [ 1358.428472] ? threaded_store+0x132/0x220 [ 1358.433769] ? ww_mutex_lock+0x160/0x160 [ 1358.438968] ? selinux_inode_free_security+0x310/0x310 [ 1358.445537] ? cap_capable+0x1ed/0x3f0 [ 1358.450543] ? sysfs_file_ops+0x140/0x140 [ 1358.455841] ? threaded_store+0x132/0x220 [ 1358.461137] threaded_store+0x132/0x220 [ 1358.466239] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.471730] ? sysfs_file_ops+0x140/0x140 [ 1358.477029] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.482326] ? sysfs_kf_write+0x65/0x160 [ 1358.487517] kernfs_fop_write_iter+0x335/0x510 [ 1358.493301] ? kernfs_ops+0xf0/0xf0 [ 1358.498012] new_sync_write+0x308/0x6f0 [ 1358.503115] ? new_sync_read+0x6f0/0x6f0 [ 1358.508315] ? lock_acquire.part.0+0xbc/0x260 [ 1358.514001] ? ksys_write+0xff/0x200 [ 1358.518813] vfs_write+0x62a/0x980 [ 1358.523428] ksys_write+0xff/0x200 [ 1358.528032] ? __ia32_sys_read+0xc0/0xc0 [ 1358.533226] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.538813] ? rcu_is_watching+0x15/0xd0 [ 1358.544014] do_syscall_64+0xf3/0x5e0 [ 1358.548922] ? exc_page_fault+0x9e/0xb0 [ 1358.554023] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.560487] RIP: 0033:0x7f7363e4dc5e [ 1358.565294] RSP: 002b:00007fffafde34f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.574593] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7363e4dc5e [ 1358.583403] RDX: 0000000000000001 RSI: 0000559eb2865110 RDI: 0000000000000007 [ 1358.592212] RBP: 00007fffafde3500 R08: 0000000000000000 R09: 0000000000000000 [ 1358.601016] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7363ce8e80 [ 1358.609826] R13: 0000559eb2865110 R14: 0000000000000007 R15: 00007f736454eb98 [ 1358.618640] [ 1358.621914] INFO: task python3:4985 is blocked on a mutex likely owned by task python3:3059. [ 1358.632183] task:python3 state:D stack:25312 pid:3059 tgid:3059 ppid:3058 task_flags:0x400100 flags:0x00080802 [ 1358.645383] Call Trace: [ 1358.648909] [ 1358.652059] __schedule+0xad9/0x1490 [ 1358.656870] ? io_schedule_timeout+0x130/0x130 [ 1358.662655] ? find_held_lock+0x2b/0x80 [ 1358.667758] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1358.673938] schedule+0xd4/0x260 [ 1358.678355] schedule_timeout+0x103/0x260 [ 1358.683651] ? rcu_is_watching+0x15/0xd0 [ 1358.688848] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1358.695219] ? hrtimers_cpu_dying+0x250/0x250 [ 1358.700911] ? ww_mutex_lock+0x160/0x160 [ 1358.706109] msleep+0x1b/0x30 [ 1358.710233] napi_set_threaded+0x376/0x410 [ 1358.715630] ? napi_resume_irqs+0x1a0/0x1a0 [ 1358.721123] netif_set_threaded+0x254/0x400 [ 1358.726617] ? sysfs_file_ops+0x140/0x140 [ 1358.731911] threaded_store+0x1bb/0x220 [ 1358.737010] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.742501] ? sysfs_file_ops+0x140/0x140 [ 1358.747794] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.753089] ? sysfs_kf_write+0x65/0x160 [ 1358.758291] kernfs_fop_write_iter+0x335/0x510 [ 1358.764078] ? kernfs_ops+0xf0/0xf0 [ 1358.765654] nipa-hw-worker: drivers/net:napi_threaded.py: timed out [ 1358.768788] new_sync_write+0x308/0x6f0 [ 1358.768794] ? new_sync_read+0x6f0/0x6f0