[ 759.775133] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 759.777676] nipa-hw-worker: [4/38] Retrying drivers/net:napi_threaded.py
[ 759.939641] kselftest: Running tests in drivers/net
[ 989.762241] INFO: task python3:5064 blocked for more than 122 seconds.
[ 989.770464] Not tainted 7.0.0-rc5-flmx-g1b317aaa646c #1
[ 989.777714] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.787304] task:python3 state:D stack:25312 pid:5064 tgid:5064 ppid:5063 task_flags:0x400100 flags:0x00080000
[ 989.800513] Call Trace:
[ 989.804052]
[ 989.807206] __schedule+0xad9/0x1490
[ 989.812023] ? io_schedule_timeout+0x130/0x130
[ 989.817810] ? find_held_lock+0x2b/0x80
[ 989.822918] ? __lock_release.isra.0+0x59/0x170
[ 989.828806] schedule+0xd4/0x260
[ 989.833226] schedule_preempt_disabled+0x18/0x30
[ 989.839197] __mutex_lock+0xd76/0x1f20
[ 989.844196] ? threaded_store+0x132/0x220
[ 989.849498] ? ww_mutex_lock+0x160/0x160
[ 989.854697] ? selinux_inode_free_security+0x310/0x310
[ 989.861270] ? cap_capable+0x1ed/0x3f0
[ 989.866281] ? sysfs_file_ops+0x140/0x140
[ 989.871583] ? threaded_store+0x132/0x220
[ 989.876882] threaded_store+0x132/0x220
[ 989.881983] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.887476] ? sysfs_file_ops+0x140/0x140
[ 989.892773] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.898073] ? sysfs_kf_write+0x65/0x160
[ 989.903273] kernfs_fop_write_iter+0x335/0x510
[ 989.909058] ? kernfs_ops+0xf0/0xf0
[ 989.913775] new_sync_write+0x308/0x6f0
[ 989.918881] ? new_sync_read+0x6f0/0x6f0
[ 989.924085] ? lock_acquire.part.0+0xbc/0x260
[ 989.929770] ? ksys_write+0xff/0x200
[ 989.934584] vfs_write+0x62a/0x980
[ 989.939197] ksys_write+0xff/0x200
[ 989.943813] ? __ia32_sys_read+0xc0/0xc0
[ 989.949012] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.954603] ? rcu_is_watching+0x15/0xd0
[ 989.959806] do_syscall_64+0xf3/0x5e0
[ 989.964713] ? exc_page_fault+0x9e/0xb0
[ 989.969817] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.976285] RIP: 0033:0x7f97ddd5fc5e
[ 989.981093] RSP: 002b:00007ffcac254220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.990391] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f97ddd5fc5e
[ 989.999190] RDX: 0000000000000001 RSI: 000055f1b94ce110 RDI: 0000000000000007
[ 990.007990] RBP: 00007ffcac254230 R08: 0000000000000000 R09: 0000000000000000
[ 990.016796] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f97ddbfae80
[ 990.025604] R13: 000055f1b94ce110 R14: 0000000000000007 R15: 00007f97de460b98
[ 990.034418]
[ 990.037690] INFO: task python3:5064 is blocked on a mutex likely owned by task python3:3105.
[ 990.047969] task:python3 state:D stack:25312 pid:3105 tgid:3105 ppid:3104 task_flags:0x400100 flags:0x00080802
[ 990.061172] Call Trace:
[ 990.064713]
[ 990.067863] __schedule+0xad9/0x1490
[ 990.072677] ? io_schedule_timeout+0x130/0x130
[ 990.078463] ? find_held_lock+0x2b/0x80
[ 990.083566] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.089749] schedule+0xd4/0x260
[ 990.094170] schedule_timeout+0x103/0x260
[ 990.099470] ? rcu_is_watching+0x15/0xd0
[ 990.104668] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.111038] ? hrtimers_cpu_dying+0x250/0x250
[ 990.116729] ? ww_mutex_lock+0x160/0x160
[ 990.121932] msleep+0x1b/0x30
[ 990.126059] napi_set_threaded+0x376/0x410
[ 990.131458] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.136954] netif_set_threaded+0x254/0x400
[ 990.142451] ? sysfs_file_ops+0x140/0x140
[ 990.147747] threaded_store+0x1bb/0x220
[ 990.152853] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.158345] ? sysfs_file_ops+0x140/0x140
[ 990.163641] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.168938] ? sysfs_kf_write+0x65/0x160
[ 990.174132] kernfs_fop_write_iter+0x335/0x510
[ 990.179915] ? kernfs_ops+0xf0/0xf0
[ 990.184625] new_sync_write+0x308/0x6f0
[ 990.189728] ? new_sync_read+0x6f0/0x6f0
[ 990.194931] ? lock_acquire.part.0+0xbc/0x260
[ 990.200618] ? ksys_write+0xff/0x200
[ 990.205423] vfs_write+0x62a/0x980
[ 990.210039] ksys_write+0xff/0x200
[ 990.214654] ? __ia32_sys_read+0xc0/0xc0
[ 990.219853] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.225442] ? rcu_is_watching+0x15/0xd0
[ 990.230643] do_syscall_64+0xf3/0x5e0
[ 990.235549] ? exc_page_fault+0x9e/0xb0
[ 990.240648] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.247114] RIP: 0033:0x7f7c9ac25c5e
[ 990.251922] RSP: 002b:00007ffcf754e840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.261221] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7c9ac25c5e
[ 990.270027] RDX: 0000000000000001 RSI: 000055d25a485110 RDI: 0000000000000007
[ 990.278836] RBP: 00007ffcf754e850 R08: 0000000000000000 R09: 0000000000000000
[ 990.287643] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7c9aac0e80
[ 990.296444] R13: 000055d25a485110 R14: 0000000000000007 R15: 00007f7c9aa4e480
[ 990.305257]
[ 990.308517] \x0aShowing all locks held in the system:
[ 990.317057] 1 lock held by khungtaskd/184:
[ 990.322452] #0: ffffffff8dece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.333918] 3 locks held by systemd-journal/805:
[ 990.339906] 4 locks held by python3/3105:
[ 990.345201] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.354995] #1: ff11000135ad0888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.365958] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.377114] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.387211] 4 locks held by python3/5064:
[ 990.392507] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.402299] #1: ff110001bb820288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.413262] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.424423] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.434514]
[ 990.436970] =============================================\x0a
[ 1051.203829] kworker/u130:2 (455) used greatest stack depth: 21192 bytes left
[ 1112.643513] INFO: task python3:5064 blocked for more than 245 seconds.
[ 1112.651668] Not tainted 7.0.0-rc5-flmx-g1b317aaa646c #1
[ 1112.658918] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.668506] task:python3 state:D stack:25312 pid:5064 tgid:5064 ppid:5063 task_flags:0x400100 flags:0x00080000
[ 1112.681713] Call Trace:
[ 1112.685255]
[ 1112.688406] __schedule+0xad9/0x1490
[ 1112.693225] ? io_schedule_timeout+0x130/0x130
[ 1112.699010] ? find_held_lock+0x2b/0x80
[ 1112.704118] ? __lock_release.isra.0+0x59/0x170
[ 1112.710006] schedule+0xd4/0x260
[ 1112.714427] schedule_preempt_disabled+0x18/0x30
[ 1112.720408] __mutex_lock+0xd76/0x1f20
[ 1112.725412] ? threaded_store+0x132/0x220
[ 1112.730713] ? ww_mutex_lock+0x160/0x160
[ 1112.735912] ? selinux_inode_free_security+0x310/0x310
[ 1112.742483] ? cap_capable+0x1ed/0x3f0
[ 1112.747491] ? sysfs_file_ops+0x140/0x140
[ 1112.752791] ? threaded_store+0x132/0x220
[ 1112.758088] threaded_store+0x132/0x220
[ 1112.763194] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.768686] ? sysfs_file_ops+0x140/0x140
[ 1112.773974] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.779271] ? sysfs_kf_write+0x65/0x160
[ 1112.784472] kernfs_fop_write_iter+0x335/0x510
[ 1112.790260] ? kernfs_ops+0xf0/0xf0
[ 1112.794973] new_sync_write+0x308/0x6f0
[ 1112.800078] ? new_sync_read+0x6f0/0x6f0
[ 1112.805270] ? lock_acquire.part.0+0xbc/0x260
[ 1112.810957] ? ksys_write+0xff/0x200
[ 1112.815768] vfs_write+0x62a/0x980
[ 1112.820385] ksys_write+0xff/0x200
[ 1112.824999] ? __ia32_sys_read+0xc0/0xc0
[ 1112.830195] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.835787] ? rcu_is_watching+0x15/0xd0
[ 1112.840989] do_syscall_64+0xf3/0x5e0
[ 1112.845894] ? exc_page_fault+0x9e/0xb0
[ 1112.850997] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.857468] RIP: 0033:0x7f97ddd5fc5e
[ 1112.862277] RSP: 002b:00007ffcac254220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.871577] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f97ddd5fc5e
[ 1112.880384] RDX: 0000000000000001 RSI: 000055f1b94ce110 RDI: 0000000000000007
[ 1112.889191] RBP: 00007ffcac254230 R08: 0000000000000000 R09: 0000000000000000
[ 1112.897999] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f97ddbfae80
[ 1112.906809] R13: 000055f1b94ce110 R14: 0000000000000007 R15: 00007f97de460b98
[ 1112.915624]
[ 1112.918951] INFO: task python3:5064 is blocked on a mutex likely owned by task python3:3105.
[ 1112.929225] task:python3 state:D stack:25312 pid:3105 tgid:3105 ppid:3104 task_flags:0x400100 flags:0x00080802
[ 1112.942428] Call Trace:
[ 1112.945965]
[ 1112.949114] __schedule+0xad9/0x1490
[ 1112.953925] ? io_schedule_timeout+0x130/0x130
[ 1112.959708] ? find_held_lock+0x2b/0x80
[ 1112.964811] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.970993] schedule+0xd4/0x260
[ 1112.975411] schedule_timeout+0x103/0x260
[ 1112.980707] ? rcu_is_watching+0x15/0xd0
[ 1112.985907] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.992277] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.997968] ? ww_mutex_lock+0x160/0x160
[ 1113.003166] msleep+0x1b/0x30
[ 1113.007295] napi_set_threaded+0x376/0x410
[ 1113.012693] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.018191] netif_set_threaded+0x254/0x400
[ 1113.023683] ? sysfs_file_ops+0x140/0x140
[ 1113.028971] threaded_store+0x1bb/0x220
[ 1113.034074] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.039567] ? sysfs_file_ops+0x140/0x140
[ 1113.044864] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.050163] ? sysfs_kf_write+0x65/0x160
[ 1113.055361] kernfs_fop_write_iter+0x335/0x510
[ 1113.061145] ? kernfs_ops+0xf0/0xf0
[ 1113.065857] new_sync_write+0x308/0x6f0
[ 1113.070962] ? new_sync_read+0x6f0/0x6f0
[ 1113.076161] ? lock_acquire.part.0+0xbc/0x260
[ 1113.081846] ? ksys_write+0xff/0x200
[ 1113.086661] vfs_write+0x62a/0x980
[ 1113.091278] ksys_write+0xff/0x200
[ 1113.095968] ? __ia32_sys_read+0xc0/0xc0
[ 1113.101168] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.106758] ? rcu_is_watching+0x15/0xd0
[ 1113.111950] do_syscall_64+0xf3/0x5e0
[ 1113.116856] ? exc_page_fault+0x9e/0xb0
[ 1113.121958] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.128426] RIP: 0033:0x7f7c9ac25c5e
[ 1113.133237] RSP: 002b:00007ffcf754e840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.142535] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7c9ac25c5e
[ 1113.151347] RDX: 0000000000000001 RSI: 000055d25a485110 RDI: 0000000000000007
[ 1113.160155] RBP: 00007ffcf754e850 R08: 0000000000000000 R09: 0000000000000000
[ 1113.168964] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7c9aac0e80
[ 1113.177772] R13: 000055d25a485110 R14: 0000000000000007 R15: 00007f7c9aa4e480
[ 1113.186587]
[ 1113.189850] \x0aShowing all locks held in the system:
[ 1113.198391] 1 lock held by khungtaskd/184:
[ 1113.203784] #0: ffffffff8dece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.215260] 3 locks held by systemd-journal/805:
[ 1113.221240] #0: ff1100180e5b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 1113.231126] #1: ffffffff8e0c91e0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.243943] #2: ff110001a31a8440 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 1113.255406] 4 locks held by python3/3105:
[ 1113.260700] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.270492] #1: ff11000135ad0888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.281454] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.292611] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.302704] 4 locks held by python3/5064:
[ 1113.307999] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.317790] #1: ff110001bb820288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.328744] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.339900] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.349991]
[ 1113.352458] =============================================\x0a
[ 1235.525769] INFO: task python3:5064 blocked for more than 368 seconds.
[ 1235.533918] Not tainted 7.0.0-rc5-flmx-g1b317aaa646c #1
[ 1235.541169] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.550757] task:python3 state:D stack:25312 pid:5064 tgid:5064 ppid:5063 task_flags:0x400100 flags:0x00080002
[ 1235.563953] Call Trace:
[ 1235.567495]
[ 1235.570646] __schedule+0xad9/0x1490
[ 1235.575466] ? io_schedule_timeout+0x130/0x130
[ 1235.581252] ? find_held_lock+0x2b/0x80
[ 1235.586358] ? __lock_release.isra.0+0x59/0x170
[ 1235.592244] schedule+0xd4/0x260
[ 1235.596662] schedule_preempt_disabled+0x18/0x30
[ 1235.602641] __mutex_lock+0xd76/0x1f20
[ 1235.607645] ? threaded_store+0x132/0x220
[ 1235.612944] ? ww_mutex_lock+0x160/0x160
[ 1235.618144] ? selinux_inode_free_security+0x310/0x310
[ 1235.624715] ? cap_capable+0x1ed/0x3f0
[ 1235.629725] ? sysfs_file_ops+0x140/0x140
[ 1235.635026] ? threaded_store+0x132/0x220
[ 1235.640322] threaded_store+0x132/0x220
[ 1235.645428] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.650921] ? sysfs_file_ops+0x140/0x140
[ 1235.656221] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.661518] ? sysfs_kf_write+0x65/0x160
[ 1235.666719] kernfs_fop_write_iter+0x335/0x510
[ 1235.672505] ? kernfs_ops+0xf0/0xf0
[ 1235.677218] new_sync_write+0x308/0x6f0
[ 1235.682323] ? new_sync_read+0x6f0/0x6f0
[ 1235.687525] ? lock_acquire.part.0+0xbc/0x260
[ 1235.693212] ? ksys_write+0xff/0x200
[ 1235.698014] vfs_write+0x62a/0x980
[ 1235.702630] ksys_write+0xff/0x200
[ 1235.707244] ? __ia32_sys_read+0xc0/0xc0
[ 1235.712445] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.718038] ? rcu_is_watching+0x15/0xd0
[ 1235.723239] do_syscall_64+0xf3/0x5e0
[ 1235.728145] ? exc_page_fault+0x9e/0xb0
[ 1235.733247] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.739718] RIP: 0033:0x7f97ddd5fc5e
[ 1235.744529] RSP: 002b:00007ffcac254220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.753827] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f97ddd5fc5e
[ 1235.762637] RDX: 0000000000000001 RSI: 000055f1b94ce110 RDI: 0000000000000007
[ 1235.771443] RBP: 00007ffcac254230 R08: 0000000000000000 R09: 0000000000000000
[ 1235.780252] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f97ddbfae80
[ 1235.789062] R13: 000055f1b94ce110 R14: 0000000000000007 R15: 00007f97de460b98
[ 1235.797878]
[ 1235.801147] INFO: task python3:5064 is blocked on a mutex likely owned by task python3:3105.
[ 1235.811422] task:python3 state:D stack:25312 pid:3105 tgid:3105 ppid:3104 task_flags:0x400100 flags:0x00080802
[ 1235.824617] Call Trace:
[ 1235.828156]
[ 1235.831308] __schedule+0xad9/0x1490
[ 1235.836118] ? io_schedule_timeout+0x130/0x130
[ 1235.841903] ? find_held_lock+0x2b/0x80
[ 1235.847007] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.853186] schedule+0xd4/0x260
[ 1235.857609] schedule_timeout+0x103/0x260
[ 1235.862906] ? rcu_is_watching+0x15/0xd0
[ 1235.868103] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.874475] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.880167] ? ww_mutex_lock+0x160/0x160
[ 1235.885367] msleep+0x1b/0x30
[ 1235.889483] napi_set_threaded+0x376/0x410
[ 1235.894881] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.900374] netif_set_threaded+0x254/0x400
[ 1235.905860] ? sysfs_file_ops+0x140/0x140
[ 1235.911155] threaded_store+0x1bb/0x220
[ 1235.916259] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.921744] ? sysfs_file_ops+0x140/0x140
[ 1235.927041] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.932331] ? sysfs_kf_write+0x65/0x160
[ 1235.937532] kernfs_fop_write_iter+0x335/0x510
[ 1235.943317] ? kernfs_ops+0xf0/0xf0
[ 1235.948031] new_sync_write+0x308/0x6f0
[ 1235.953132] ? new_sync_read+0x6f0/0x6f0
[ 1235.958334] ? lock_acquire.part.0+0xbc/0x260
[ 1235.964020] ? ksys_write+0xff/0x200
[ 1235.968831] vfs_write+0x62a/0x980
[ 1235.973446] ksys_write+0xff/0x200
[ 1235.978064] ? __ia32_sys_read+0xc0/0xc0
[ 1235.983261] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.988854] ? rcu_is_watching+0x15/0xd0
[ 1235.994053] do_syscall_64+0xf3/0x5e0
[ 1235.998958] ? exc_page_fault+0x9e/0xb0
[ 1236.004059] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1236.010528] RIP: 0033:0x7f7c9ac25c5e
[ 1236.015336] RSP: 002b:00007ffcf754e840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.024633] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7c9ac25c5e
[ 1236.033434] RDX: 0000000000000001 RSI: 000055d25a485110 RDI: 0000000000000007
[ 1236.042243] RBP: 00007ffcf754e850 R08: 0000000000000000 R09: 0000000000000000
[ 1236.051054] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7c9aac0e80
[ 1236.059862] R13: 000055d25a485110 R14: 0000000000000007 R15: 00007f7c9aa4e480
[ 1236.068677]
[ 1236.071941] \x0aShowing all locks held in the system:
[ 1236.080479] 1 lock held by khungtaskd/184:
[ 1236.085875] #0: ffffffff8dece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.097349] 2 locks held by systemd-journal/805:
[ 1236.103327] #0: ff1100180e6372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 1236.113214] #1: ff110001a0321c00 (&mm->mmap_lock){++++}-{4:4}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.123994] 4 locks held by python3/3105:
[ 1236.129289] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.139081] #1: ff11000135ad0888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.150035] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.161193] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.171284] 4 locks held by python3/5064:
[ 1236.176582] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.186372] #1: ff110001bb820288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.197335] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.208493] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.218583]
[ 1236.221051] =============================================\x0a
[ 1358.407107] INFO: task python3:5064 blocked for more than 491 seconds.
[ 1358.415257] Not tainted 7.0.0-rc5-flmx-g1b317aaa646c #1
[ 1358.422507] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.432099] task:python3 state:D stack:25312 pid:5064 tgid:5064 ppid:5063 task_flags:0x400100 flags:0x00080002
[ 1358.445306] Call Trace:
[ 1358.448847]
[ 1358.451998] __schedule+0xad9/0x1490
[ 1358.456814] ? io_schedule_timeout+0x130/0x130
[ 1358.462603] ? find_held_lock+0x2b/0x80
[ 1358.467711] ? __lock_release.isra.0+0x59/0x170
[ 1358.473595] schedule+0xd4/0x260
[ 1358.478011] schedule_preempt_disabled+0x18/0x30
[ 1358.483988] __mutex_lock+0xd76/0x1f20
[ 1358.488992] ? threaded_store+0x132/0x220
[ 1358.494290] ? ww_mutex_lock+0x160/0x160
[ 1358.499489] ? selinux_inode_free_security+0x310/0x310
[ 1358.506059] ? cap_capable+0x1ed/0x3f0
[ 1358.511067] ? sysfs_file_ops+0x140/0x140
[ 1358.516366] ? threaded_store+0x132/0x220
[ 1358.521659] threaded_store+0x132/0x220
[ 1358.526763] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.532256] ? sysfs_file_ops+0x140/0x140
[ 1358.537542] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.542837] ? sysfs_kf_write+0x65/0x160
[ 1358.548037] kernfs_fop_write_iter+0x335/0x510
[ 1358.553824] ? kernfs_ops+0xf0/0xf0
[ 1358.558533] new_sync_write+0x308/0x6f0
[ 1358.563635] ? new_sync_read+0x6f0/0x6f0
[ 1358.568835] ? lock_acquire.part.0+0xbc/0x260
[ 1358.574512] ? ksys_write+0xff/0x200
[ 1358.579323] vfs_write+0x62a/0x980
[ 1358.583942] ksys_write+0xff/0x200
[ 1358.588556] ? __ia32_sys_read+0xc0/0xc0
[ 1358.593752] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.599340] ? rcu_is_watching+0x15/0xd0
[ 1358.604541] do_syscall_64+0xf3/0x5e0
[ 1358.609447] ? exc_page_fault+0x9e/0xb0
[ 1358.614549] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.621016] RIP: 0033:0x7f97ddd5fc5e
[ 1358.625818] RSP: 002b:00007ffcac254220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.635116] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f97ddd5fc5e
[ 1358.643922] RDX: 0000000000000001 RSI: 000055f1b94ce110 RDI: 0000000000000007
[ 1358.652732] RBP: 00007ffcac254230 R08: 0000000000000000 R09: 0000000000000000
[ 1358.661541] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f97ddbfae80
[ 1358.670347] R13: 000055f1b94ce110 R14: 0000000000000007 R15: 00007f97de460b98
[ 1358.679158]
[ 1358.682456] INFO: task python3:5064 is blocked on a mutex likely owned by task python3:3105.
[ 1358.692731] task:python3 state:D stack:25312 pid:3105 tgid:3105 ppid:3104 task_flags:0x400100 flags:0x00080802
[ 1358.705932] Call Trace:
[ 1358.709474]
[ 1358.712621] __schedule+0xad9/0x1490
[ 1358.717435] ? io_schedule_timeout+0x130/0x130
[ 1358.723218] ? find_held_lock+0x2b/0x80
[ 1358.728386] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.734567] schedule+0xd4/0x260
[ 1358.738986] schedule_timeout+0x103/0x260
[ 1358.744274] ? rcu_is_watching+0x15/0xd0
[ 1358.749545] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.755915] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.761600] ? ww_mutex_lock+0x160/0x160
[ 1358.761606] msleep+0x1b/0x30
[ 1358.761610] napi_set_threaded+0x376/0x410
[ 1358.761615] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.781862] netif_set_threaded+0x254/0x400
[ 1358.787357] ? sysfs_file_ops+0x140/0x140
[ 1358.792653] threaded_store+0x1bb/0x220
[ 1358.797758] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.803250] ? sysfs_file_ops+0x140/0x140
[ 1358.808549] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.813846] ? sysfs_kf_write+0x65/0x160
[ 1358.819049] kernfs_fop_write_iter+0x335/0x510
[ 1358.824835] ? kernfs_ops+0xf0/0xf0
[ 1358.829550] new_sync_write+0x308/0x6f0
[ 1358.834652] ? new_sync_read+0x6f0/0x6f0
[ 1358.839846] ? lock_acquire.part.0+0xbc/0x260
[ 1358.845533] ? ksys_write+0xff/0x200
[ 1358.850345] vfs_write+0x62a/0x980
[ 1358.854962] ksys_write+0xff/0x200
[ 1358.859577] ? __ia32_sys_read+0xc0/0xc0
[ 1358.864776] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.870359] ? rcu_is_watching+0x15/0xd0
[ 1358.875559] do_syscall_64+0xf3/0x5e0
[ 1358.880464] ? exc_page_fault+0x9e/0xb0
[ 1358.885568] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.892033] RIP: 0033:0x7f7c9ac25c5e
[ 1358.896840] RSP: 002b:00007ffcf754e840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.906137] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7c9ac25c5e
[ 1358.914951] RDX: 0000000000000001 RSI: 000055d25a485110 RDI: 0000000000000007
[ 1358.923764] RBP: 00007ffcf754e850 R08: 0000000000000000 R09: 0000000000000000
[ 1358.932575] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7c9aac0e80
[ 1358.941381] R13: 000055d25a485110 R14: 0000000000000007 R15: 00007f7c9aa4e480
[ 1358.950197]
[ 1358.953466] \x0aShowing all locks held in the system:
[ 1358.962009] 1 lock held by khungtaskd/184:
[ 1358.967392] #0: ffffffff8dece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.978875] 4 locks held by python3/3105:
[ 1358.984171] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.993965] #1: ff11000135ad0888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.004927] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.016086] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.026182] 4 locks held by python3/5064:
[ 1359.031477] #0: ff1100012da893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.041266] #1: ff110001bb820288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.052229] #2: ff11000120ad10f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.063388] #3: ff11000120aa9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.073483] 5 locks held by (time-dir)/6919:
[ 1359.079071] #0: ff1100012dba93f8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.088863] #1: ff1100011c14ac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.099824] #2: ffffffff8df37248 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xd2/0x3b0
[ 1359.110495] #3: ffffffff8dd68f30 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e7/0x640
[ 1359.122140] #4: ffffffff8df36f50 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f3/0x640
[ 1359.134566]
[ 1359.137036] =============================================\x0a