[ 740.352274] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 740.356365] nipa-hw-worker: [4/36] Retrying drivers/net:napi_threaded.py
[ 740.520051] kselftest: Running tests in drivers/net
[ 862.749129] kworker/u130:4 (529) used greatest stack depth: 21896 bytes left
[ 866.844157] INFO: task python3:4687 blocked for more than 122 seconds.
[ 866.852389] Not tainted 7.0.0-rc3-crws-gdc41423dc228 #1
[ 866.859636] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 866.869223] task:python3 state:D stack:25312 pid:4687 tgid:4687 ppid:4686 task_flags:0x400100 flags:0x00080000
[ 866.882427] Call Trace:
[ 866.885963]
[ 866.889114] __schedule+0xad9/0x1490
[ 866.893926] ? io_schedule_timeout+0x130/0x130
[ 866.899704] ? find_held_lock+0x2b/0x80
[ 866.904807] ? __lock_release.isra.0+0x59/0x170
[ 866.910694] schedule+0xd4/0x260
[ 866.915111] schedule_preempt_disabled+0x18/0x30
[ 866.921089] __mutex_lock+0xd76/0x1f20
[ 866.926094] ? threaded_store+0x132/0x220
[ 866.931393] ? ww_mutex_lock+0x160/0x160
[ 866.936593] ? selinux_inode_free_security+0x310/0x310
[ 866.943163] ? cap_capable+0x1ed/0x3f0
[ 866.948171] ? sysfs_file_ops+0x140/0x140
[ 866.953473] ? threaded_store+0x132/0x220
[ 866.958767] threaded_store+0x132/0x220
[ 866.963869] ? tx_maxrate_store+0x4e0/0x4e0
[ 866.969360] ? sysfs_file_ops+0x140/0x140
[ 866.974655] ? sysfs_file_kobj+0xbd/0x1e0
[ 866.979951] ? sysfs_kf_write+0x65/0x160
[ 866.985149] kernfs_fop_write_iter+0x335/0x510
[ 866.990931] ? kernfs_ops+0xf0/0xf0
[ 866.995643] new_sync_write+0x308/0x6f0
[ 867.000745] ? new_sync_read+0x6f0/0x6f0
[ 867.005946] ? lock_acquire.part.0+0xbc/0x260
[ 867.011633] ? ksys_write+0xff/0x200
[ 867.016445] vfs_write+0x62a/0x980
[ 867.021059] ksys_write+0xff/0x200
[ 867.025672] ? __ia32_sys_read+0xc0/0xc0
[ 867.030869] ? kernfs_fop_llseek+0x160/0x1f0
[ 867.036457] ? rcu_is_watching+0x15/0xd0
[ 867.041660] do_syscall_64+0xf3/0x5e0
[ 867.046563] ? exc_page_fault+0x9e/0xb0
[ 867.051662] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 867.058128] RIP: 0033:0x7fdb7c1dac5e
[ 867.062936] RSP: 002b:00007ffc2409e890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 867.072232] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fdb7c1dac5e
[ 867.081039] RDX: 0000000000000001 RSI: 000055774b178cd0 RDI: 0000000000000007
[ 867.089846] RBP: 00007ffc2409e8a0 R08: 0000000000000000 R09: 0000000000000000
[ 867.098644] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fdb7c075e80
[ 867.107451] R13: 000055774b178cd0 R14: 0000000000000007 R15: 00007fdb7c8dbb98
[ 867.116264]
[ 867.119544] INFO: task python3:4687 is blocked on a mutex likely owned by task python3:2716.
[ 867.129816] task:python3 state:D stack:25312 pid:2716 tgid:2716 ppid:2715 task_flags:0x400100 flags:0x00080802
[ 867.143019] Call Trace:
[ 867.146557]
[ 867.149706] __schedule+0xad9/0x1490
[ 867.154514] ? io_schedule_timeout+0x130/0x130
[ 867.160296] ? find_held_lock+0x2b/0x80
[ 867.165398] ? __try_to_del_timer_sync+0x154/0x1f0
[ 867.171565] schedule+0xd4/0x260
[ 867.175987] schedule_timeout+0x10d/0x260
[ 867.181280] ? rcu_is_watching+0x15/0xd0
[ 867.186478] ? hrtimer_nanosleep_restart+0x180/0x180
[ 867.192846] ? hrtimers_cpu_dying+0x250/0x250
[ 867.198536] ? ww_mutex_lock+0x160/0x160
[ 867.203732] msleep+0x1b/0x30
[ 867.207855] napi_set_threaded+0x376/0x410
[ 867.213250] ? napi_resume_irqs+0x1a0/0x1a0
[ 867.218744] netif_set_threaded+0x254/0x400
[ 867.224236] ? sysfs_file_ops+0x140/0x140
[ 867.229533] threaded_store+0x1bb/0x220
[ 867.234632] ? tx_maxrate_store+0x4e0/0x4e0
[ 867.240124] ? sysfs_file_ops+0x140/0x140
[ 867.245409] ? sysfs_file_kobj+0xbd/0x1e0
[ 867.250705] ? sysfs_kf_write+0x65/0x160
[ 867.255904] kernfs_fop_write_iter+0x335/0x510
[ 867.261686] ? kernfs_ops+0xf0/0xf0
[ 867.266397] new_sync_write+0x308/0x6f0
[ 867.271500] ? new_sync_read+0x6f0/0x6f0
[ 867.276688] ? lock_acquire.part.0+0xbc/0x260
[ 867.282374] ? ksys_write+0xff/0x200
[ 867.287183] vfs_write+0x62a/0x980
[ 867.291797] ksys_write+0xff/0x200
[ 867.296410] ? __ia32_sys_read+0xc0/0xc0
[ 867.301610] ? kernfs_fop_llseek+0x160/0x1f0
[ 867.307190] ? rcu_is_watching+0x15/0xd0
[ 867.312388] do_syscall_64+0xf3/0x5e0
[ 867.317293] ? exc_page_fault+0x9e/0xb0
[ 867.322393] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 867.328860] RIP: 0033:0x7f506b67ac5e
[ 867.333667] RSP: 002b:00007fff6ed85a00 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 867.342961] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f506b67ac5e
[ 867.351766] RDX: 0000000000000001 RSI: 000055c074624cd0 RDI: 0000000000000007
[ 867.360574] RBP: 00007fff6ed85a10 R08: 0000000000000000 R09: 0000000000000000
[ 867.369383] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f506b515e80
[ 867.378190] R13: 000055c074624cd0 R14: 0000000000000007 R15: 00007f506b4a6480
[ 867.387002]
[ 867.390255] \x0aShowing all locks held in the system:
[ 867.398793] 1 lock held by khungtaskd/185:
[ 867.404187] #0: ffffffff8aecde20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 867.415666] 4 locks held by python3/2716:
[ 867.420959] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 867.430750] #1: ff110001edb29488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 867.441710] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 867.452863] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 867.462963] 4 locks held by python3/4687:
[ 867.468256] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 867.478044] #1: ff11000158f30888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 867.489005] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 867.500155] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 867.510244]
[ 867.512708] =============================================\x0a
[ 989.726381] INFO: task python3:4687 blocked for more than 245 seconds.
[ 989.734532] Not tainted 7.0.0-rc3-crws-gdc41423dc228 #1
[ 989.741786] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.751377] task:python3 state:D stack:25312 pid:4687 tgid:4687 ppid:4686 task_flags:0x400100 flags:0x00080002
[ 989.764583] Call Trace:
[ 989.768122]
[ 989.771272] __schedule+0xad9/0x1490
[ 989.776087] ? io_schedule_timeout+0x130/0x130
[ 989.781872] ? find_held_lock+0x2b/0x80
[ 989.786979] ? __lock_release.isra.0+0x59/0x170
[ 989.792870] schedule+0xd4/0x260
[ 989.797291] schedule_preempt_disabled+0x18/0x30
[ 989.803272] __mutex_lock+0xd76/0x1f20
[ 989.808280] ? threaded_store+0x132/0x220
[ 989.813580] ? ww_mutex_lock+0x160/0x160
[ 989.818778] ? selinux_inode_free_security+0x310/0x310
[ 989.825351] ? cap_capable+0x1ed/0x3f0
[ 989.830359] ? sysfs_file_ops+0x140/0x140
[ 989.835660] ? threaded_store+0x132/0x220
[ 989.840956] threaded_store+0x132/0x220
[ 989.846058] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.851549] ? sysfs_file_ops+0x140/0x140
[ 989.856845] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.862144] ? sysfs_kf_write+0x65/0x160
[ 989.867342] kernfs_fop_write_iter+0x335/0x510
[ 989.873126] ? kernfs_ops+0xf0/0xf0
[ 989.877838] new_sync_write+0x308/0x6f0
[ 989.882943] ? new_sync_read+0x6f0/0x6f0
[ 989.888141] ? lock_acquire.part.0+0xbc/0x260
[ 989.893831] ? ksys_write+0xff/0x200
[ 989.898642] vfs_write+0x62a/0x980
[ 989.903256] ksys_write+0xff/0x200
[ 989.907871] ? __ia32_sys_read+0xc0/0xc0
[ 989.913070] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.918658] ? rcu_is_watching+0x15/0xd0
[ 989.923863] do_syscall_64+0xf3/0x5e0
[ 989.928770] ? exc_page_fault+0x9e/0xb0
[ 989.933865] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.940333] RIP: 0033:0x7fdb7c1dac5e
[ 989.945144] RSP: 002b:00007ffc2409e890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.954444] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fdb7c1dac5e
[ 989.963254] RDX: 0000000000000001 RSI: 000055774b178cd0 RDI: 0000000000000007
[ 989.972062] RBP: 00007ffc2409e8a0 R08: 0000000000000000 R09: 0000000000000000
[ 989.980861] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fdb7c075e80
[ 989.989671] R13: 000055774b178cd0 R14: 0000000000000007 R15: 00007fdb7c8dbb98
[ 989.998491]
[ 990.001767] INFO: task python3:4687 is blocked on a mutex likely owned by task python3:2716.
[ 990.012038] task:python3 state:D stack:25312 pid:2716 tgid:2716 ppid:2715 task_flags:0x400100 flags:0x00080802
[ 990.025239] Call Trace:
[ 990.028779]
[ 990.031930] __schedule+0xad9/0x1490
[ 990.036735] ? io_schedule_timeout+0x130/0x130
[ 990.042519] ? find_held_lock+0x2b/0x80
[ 990.047622] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.053803] schedule+0xd4/0x260
[ 990.058221] schedule_timeout+0x103/0x260
[ 990.063521] ? rcu_is_watching+0x15/0xd0
[ 990.068720] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.075089] ? hrtimers_cpu_dying+0x250/0x250
[ 990.080771] ? ww_mutex_lock+0x160/0x160
[ 990.085970] msleep+0x1b/0x30
[ 990.090094] napi_set_threaded+0x376/0x410
[ 990.095490] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.100984] netif_set_threaded+0x254/0x400
[ 990.106477] ? sysfs_file_ops+0x140/0x140
[ 990.111767] threaded_store+0x1bb/0x220
[ 990.116871] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.122364] ? sysfs_file_ops+0x140/0x140
[ 990.127661] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.132960] ? sysfs_kf_write+0x65/0x160
[ 990.138160] kernfs_fop_write_iter+0x335/0x510
[ 990.143947] ? kernfs_ops+0xf0/0xf0
[ 990.148657] new_sync_write+0x308/0x6f0
[ 990.153759] ? new_sync_read+0x6f0/0x6f0
[ 990.158957] ? lock_acquire.part.0+0xbc/0x260
[ 990.164643] ? ksys_write+0xff/0x200
[ 990.169454] vfs_write+0x62a/0x980
[ 990.174072] ksys_write+0xff/0x200
[ 990.178687] ? __ia32_sys_read+0xc0/0xc0
[ 990.183887] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.189468] ? rcu_is_watching+0x15/0xd0
[ 990.194661] do_syscall_64+0xf3/0x5e0
[ 990.199569] ? exc_page_fault+0x9e/0xb0
[ 990.204674] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.211142] RIP: 0033:0x7f506b67ac5e
[ 990.215948] RSP: 002b:00007fff6ed85a00 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.225245] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f506b67ac5e
[ 990.234044] RDX: 0000000000000001 RSI: 000055c074624cd0 RDI: 0000000000000007
[ 990.242852] RBP: 00007fff6ed85a10 R08: 0000000000000000 R09: 0000000000000000
[ 990.251662] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f506b515e80
[ 990.260468] R13: 000055c074624cd0 R14: 0000000000000007 R15: 00007f506b4a6480
[ 990.269283]
[ 990.272544] \x0aShowing all locks held in the system:
[ 990.281063] 2 locks held by systemd/1:
[ 990.286068] #0: ff1100012e3a13f8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.295862] #1: ff11000184f1e888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.306837] 1 lock held by khungtaskd/185:
[ 990.312232] #0: ffffffff8aecde20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.323703] 3 locks held by systemd-journal/797:
[ 990.329681] #0: ff1100180e6b71e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 990.339569] #1: ffffffff8b0c8960 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.352384] #2: ff11000191a73990 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 990.363844] 4 locks held by python3/2716:
[ 990.369141] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.378931] #1: ff110001edb29488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.389893] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.401050] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.411143] 4 locks held by python3/4687:
[ 990.416437] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.426229] #1: ff11000158f30888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.437190] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.448345] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.458436]
[ 990.460902] =============================================\x0a
[ 1112.606588] INFO: task python3:4687 blocked for more than 368 seconds.
[ 1112.614738] Not tainted 7.0.0-rc3-crws-gdc41423dc228 #1
[ 1112.621988] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.631580] task:python3 state:D stack:25312 pid:4687 tgid:4687 ppid:4686 task_flags:0x400100 flags:0x00080002
[ 1112.644788] Call Trace:
[ 1112.648329]
[ 1112.651479] __schedule+0xad9/0x1490
[ 1112.656295] ? io_schedule_timeout+0x130/0x130
[ 1112.662082] ? find_held_lock+0x2b/0x80
[ 1112.667187] ? __lock_release.isra.0+0x59/0x170
[ 1112.673071] schedule+0xd4/0x260
[ 1112.677481] schedule_preempt_disabled+0x18/0x30
[ 1112.683461] __mutex_lock+0xd76/0x1f20
[ 1112.688470] ? threaded_store+0x132/0x220
[ 1112.693771] ? ww_mutex_lock+0x160/0x160
[ 1112.698968] ? selinux_inode_free_security+0x310/0x310
[ 1112.705536] ? cap_capable+0x1ed/0x3f0
[ 1112.710544] ? sysfs_file_ops+0x140/0x140
[ 1112.715845] ? threaded_store+0x132/0x220
[ 1112.721145] threaded_store+0x132/0x220
[ 1112.726325] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.731811] ? sysfs_file_ops+0x140/0x140
[ 1112.737109] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.742406] ? sysfs_kf_write+0x65/0x160
[ 1112.747675] kernfs_fop_write_iter+0x335/0x510
[ 1112.753454] ? kernfs_ops+0xf0/0xf0
[ 1112.758166] new_sync_write+0x308/0x6f0
[ 1112.763268] ? new_sync_read+0x6f0/0x6f0
[ 1112.768470] ? lock_acquire.part.0+0xbc/0x260
[ 1112.774226] ? ksys_write+0xff/0x200
[ 1112.779042] vfs_write+0x62a/0x980
[ 1112.783659] ksys_write+0xff/0x200
[ 1112.788275] ? __ia32_sys_read+0xc0/0xc0
[ 1112.793474] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.799057] ? rcu_is_watching+0x15/0xd0
[ 1112.804261] do_syscall_64+0xf3/0x5e0
[ 1112.809169] ? exc_page_fault+0x9e/0xb0
[ 1112.814274] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.820744] RIP: 0033:0x7fdb7c1dac5e
[ 1112.825554] RSP: 002b:00007ffc2409e890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.834852] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fdb7c1dac5e
[ 1112.843662] RDX: 0000000000000001 RSI: 000055774b178cd0 RDI: 0000000000000007
[ 1112.852470] RBP: 00007ffc2409e8a0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.861280] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fdb7c075e80
[ 1112.870089] R13: 000055774b178cd0 R14: 0000000000000007 R15: 00007fdb7c8dbb98
[ 1112.878903]
[ 1112.882206] INFO: task python3:4687 is blocked on a mutex likely owned by task python3:2716.
[ 1112.892479] task:python3 state:D stack:25312 pid:2716 tgid:2716 ppid:2715 task_flags:0x400100 flags:0x00080802
[ 1112.905680] Call Trace:
[ 1112.909219]
[ 1112.912371] __schedule+0xad9/0x1490
[ 1112.917184] ? io_schedule_timeout+0x130/0x130
[ 1112.922967] ? find_held_lock+0x2b/0x80
[ 1112.928072] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.934252] schedule+0xd4/0x260
[ 1112.938671] schedule_timeout+0x103/0x260
[ 1112.943968] ? rcu_is_watching+0x15/0xd0
[ 1112.949166] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.955536] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.961227] ? ww_mutex_lock+0x160/0x160
[ 1112.966430] msleep+0x1b/0x30
[ 1112.970554] napi_set_threaded+0x376/0x410
[ 1112.975952] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.981448] netif_set_threaded+0x254/0x400
[ 1112.986942] ? sysfs_file_ops+0x140/0x140
[ 1112.992242] threaded_store+0x1bb/0x220
[ 1112.997347] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.002842] ? sysfs_file_ops+0x140/0x140
[ 1113.008138] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.013436] ? sysfs_kf_write+0x65/0x160
[ 1113.018636] kernfs_fop_write_iter+0x335/0x510
[ 1113.024420] ? kernfs_ops+0xf0/0xf0
[ 1113.029132] new_sync_write+0x308/0x6f0
[ 1113.034235] ? new_sync_read+0x6f0/0x6f0
[ 1113.039439] ? lock_acquire.part.0+0xbc/0x260
[ 1113.045125] ? ksys_write+0xff/0x200
[ 1113.049938] vfs_write+0x62a/0x980
[ 1113.054555] ksys_write+0xff/0x200
[ 1113.059171] ? __ia32_sys_read+0xc0/0xc0
[ 1113.064369] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.069958] ? rcu_is_watching+0x15/0xd0
[ 1113.075159] do_syscall_64+0xf3/0x5e0
[ 1113.080066] ? exc_page_fault+0x9e/0xb0
[ 1113.085170] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.091637] RIP: 0033:0x7f506b67ac5e
[ 1113.096446] RSP: 002b:00007fff6ed85a00 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.105746] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f506b67ac5e
[ 1113.114552] RDX: 0000000000000001 RSI: 000055c074624cd0 RDI: 0000000000000007
[ 1113.123360] RBP: 00007fff6ed85a10 R08: 0000000000000000 R09: 0000000000000000
[ 1113.132166] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f506b515e80
[ 1113.140972] R13: 000055c074624cd0 R14: 0000000000000007 R15: 00007f506b4a6480
[ 1113.149788]
[ 1113.153054] \x0aShowing all locks held in the system:
[ 1113.161584] 1 lock held by khungtaskd/185:
[ 1113.166979] #0: ffffffff8aecde20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.178448] 3 locks held by systemd-journal/797:
[ 1113.184424] #0: ff1100180e7371e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 1113.194320] #1: ffffffff8b0c8960 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.207140] #2: ff11000191a73990 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 1113.218599] 4 locks held by python3/2716:
[ 1113.223894] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.233686] #1: ff110001edb29488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.244649] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.255808] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.265903] 4 locks held by python3/4687:
[ 1113.271202] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.280994] #1: ff11000158f30888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.291957] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.303115] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.313205]
[ 1113.315671] =============================================\x0a
[ 1235.488830] INFO: task python3:4687 blocked for more than 491 seconds.
[ 1235.496979] Not tainted 7.0.0-rc3-crws-gdc41423dc228 #1
[ 1235.504229] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.513821] task:python3 state:D stack:25312 pid:4687 tgid:4687 ppid:4686 task_flags:0x400100 flags:0x00080002
[ 1235.527029] Call Trace:
[ 1235.530569]
[ 1235.533723] __schedule+0xad9/0x1490
[ 1235.538541] ? io_schedule_timeout+0x130/0x130
[ 1235.544329] ? find_held_lock+0x2b/0x80
[ 1235.549436] ? __lock_release.isra.0+0x59/0x170
[ 1235.555322] schedule+0xd4/0x260
[ 1235.559739] schedule_preempt_disabled+0x18/0x30
[ 1235.565720] __mutex_lock+0xd76/0x1f20
[ 1235.570724] ? threaded_store+0x132/0x220
[ 1235.576021] ? ww_mutex_lock+0x160/0x160
[ 1235.581221] ? selinux_inode_free_security+0x310/0x310
[ 1235.587790] ? cap_capable+0x1ed/0x3f0
[ 1235.592799] ? sysfs_file_ops+0x140/0x140
[ 1235.598100] ? threaded_store+0x132/0x220
[ 1235.603396] threaded_store+0x132/0x220
[ 1235.608498] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.613991] ? sysfs_file_ops+0x140/0x140
[ 1235.619291] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.624587] ? sysfs_kf_write+0x65/0x160
[ 1235.629788] kernfs_fop_write_iter+0x335/0x510
[ 1235.635574] ? kernfs_ops+0xf0/0xf0
[ 1235.640286] new_sync_write+0x308/0x6f0
[ 1235.645388] ? new_sync_read+0x6f0/0x6f0
[ 1235.650589] ? lock_acquire.part.0+0xbc/0x260
[ 1235.656277] ? ksys_write+0xff/0x200
[ 1235.661088] vfs_write+0x62a/0x980
[ 1235.665704] ksys_write+0xff/0x200
[ 1235.670318] ? __ia32_sys_read+0xc0/0xc0
[ 1235.675515] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.681105] ? rcu_is_watching+0x15/0xd0
[ 1235.686307] do_syscall_64+0xf3/0x5e0
[ 1235.691216] ? exc_page_fault+0x9e/0xb0
[ 1235.696319] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.702788] RIP: 0033:0x7fdb7c1dac5e
[ 1235.707600] RSP: 002b:00007ffc2409e890 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.716896] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fdb7c1dac5e
[ 1235.725706] RDX: 0000000000000001 RSI: 000055774b178cd0 RDI: 0000000000000007
[ 1235.734514] RBP: 00007ffc2409e8a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.743313] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fdb7c075e80
[ 1235.752113] R13: 000055774b178cd0 R14: 0000000000000007 R15: 00007fdb7c8dbb98
[ 1235.760929]
[ 1235.764191] INFO: task python3:4687 is blocked on a mutex likely owned by task python3:2716.
[ 1235.774461] task:python3 state:D stack:25312 pid:2716 tgid:2716 ppid:2715 task_flags:0x400100 flags:0x00080802
[ 1235.787669] Call Trace:
[ 1235.791200]
[ 1235.794352] __schedule+0xad9/0x1490
[ 1235.799164] ? io_schedule_timeout+0x130/0x130
[ 1235.804950] ? find_held_lock+0x2b/0x80
[ 1235.810051] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.816231] schedule+0xd4/0x260
[ 1235.820649] schedule_timeout+0x103/0x260
[ 1235.825946] ? rcu_is_watching+0x15/0xd0
[ 1235.831148] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.837518] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.843211] ? ww_mutex_lock+0x160/0x160
[ 1235.848410] msleep+0x1b/0x30
[ 1235.852527] napi_set_threaded+0x376/0x410
[ 1235.857921] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.863414] netif_set_threaded+0x254/0x400
[ 1235.868909] ? sysfs_file_ops+0x140/0x140
[ 1235.874204] threaded_store+0x1bb/0x220
[ 1235.879307] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.884797] ? sysfs_file_ops+0x140/0x140
[ 1235.890095] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.895391] ? sysfs_kf_write+0x65/0x160
[ 1235.900591] kernfs_fop_write_iter+0x335/0x510
[ 1235.906378] ? kernfs_ops+0xf0/0xf0
[ 1235.911091] new_sync_write+0x308/0x6f0
[ 1235.916183] ? new_sync_read+0x6f0/0x6f0
[ 1235.921385] ? lock_acquire.part.0+0xbc/0x260
[ 1235.927072] ? ksys_write+0xff/0x200
[ 1235.931883] vfs_write+0x62a/0x980
[ 1235.936498] ksys_write+0xff/0x200
[ 1235.941106] ? __ia32_sys_read+0xc0/0xc0
[ 1235.946308] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.951900] ? rcu_is_watching+0x15/0xd0
[ 1235.957098] do_syscall_64+0xf3/0x5e0
[ 1235.962003] ? exc_page_fault+0x9e/0xb0
[ 1235.967107] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.973575] RIP: 0033:0x7f506b67ac5e
[ 1235.978385] RSP: 002b:00007fff6ed85a00 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.987676] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f506b67ac5e
[ 1235.996482] RDX: 0000000000000001 RSI: 000055c074624cd0 RDI: 0000000000000007
[ 1236.005290] RBP: 00007fff6ed85a10 R08: 0000000000000000 R09: 0000000000000000
[ 1236.014102] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f506b515e80
[ 1236.022911] R13: 000055c074624cd0 R14: 0000000000000007 R15: 00007f506b4a6480
[ 1236.031727]
[ 1236.034992] \x0aShowing all locks held in the system:
[ 1236.043530] 1 lock held by khungtaskd/185:
[ 1236.048926] #0: ffffffff8aecde20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.060409] 4 locks held by python3/2716:
[ 1236.065705] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.075499] #1: ff110001edb29488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.086463] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.097621] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.107712] 4 locks held by python3/4687:
[ 1236.112997] #0: ff1100012e2b13f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.122787] #1: ff11000158f30888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.133749] #2: ff110001223090f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.144906] #3: ff110001222e9cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.154998]
[ 1236.157465] =============================================\x0a