[ 781.313749] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 781.316135] nipa-hw-worker: [4/40] Retrying drivers/net:napi_threaded.py
[ 781.468689] kselftest: Running tests in drivers/net
[ 989.655604] INFO: task python3:5617 blocked for more than 122 seconds.
[ 989.663826] Not tainted 7.0.0-rc6-elmk-g80a4209001ba #1
[ 989.671076] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.680668] task:python3 state:D stack:25336 pid:5617 tgid:5617 ppid:5616 task_flags:0x400100 flags:0x00080000
[ 989.693873] Call Trace:
[ 989.697412]
[ 989.700565] __schedule+0xae3/0x14d0
[ 989.705383] ? io_schedule_timeout+0x140/0x140
[ 989.711170] ? find_held_lock+0x2b/0x80
[ 989.716275] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.722163] schedule+0xd4/0x260
[ 989.726584] schedule_preempt_disabled+0x18/0x30
[ 989.732566] __mutex_lock+0x6b2/0x1ee0
[ 989.737574] ? rcu_is_watching+0x15/0xd0
[ 989.742774] ? threaded_store+0x132/0x220
[ 989.748073] ? ww_mutex_lock+0x160/0x160
[ 989.753262] ? selinux_inode_free_security+0x330/0x330
[ 989.759832] ? cap_capable+0x1ed/0x3f0
[ 989.764840] ? threaded_store+0x132/0x220
[ 989.770137] threaded_store+0x132/0x220
[ 989.775235] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.780534] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.785831] ? sysfs_kf_write+0x65/0x160
[ 989.791030] ? sysfs_file_ops+0x140/0x140
[ 989.796324] kernfs_fop_write_iter+0x335/0x510
[ 989.802112] ? kernfs_ops+0xf0/0xf0
[ 989.806823] new_sync_write+0x333/0x750
[ 989.811927] ? __lock_acquire+0x508/0xc10
[ 989.817222] ? new_sync_read+0x740/0x740
[ 989.822422] ? lock_acquire.part.0+0xbc/0x260
[ 989.828109] ? ksys_write+0x116/0x250
[ 989.833022] vfs_write+0x657/0x9b0
[ 989.837639] ksys_write+0x116/0x250
[ 989.842352] ? __ia32_sys_read+0xc0/0xc0
[ 989.847553] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.853145] ? rcu_is_watching+0x15/0xd0
[ 989.858346] do_syscall_64+0xf3/0x5e0
[ 989.863253] ? irq_exit_rcu+0x39/0xf0
[ 989.868155] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.874623] RIP: 0033:0x7f6887b1b22e
[ 989.879437] RSP: 002b:00007fffdba8f830 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.888735] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6887b1b22e
[ 989.897546] RDX: 0000000000000001 RSI: 0000558f9b964f40 RDI: 0000000000000007
[ 989.906355] RBP: 00007fffdba8f840 R08: 0000000000000000 R09: 0000000000000000
[ 989.915162] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6887993390
[ 989.923962] R13: 0000558f9b964f40 R14: 0000000000000007 R15: 0000000000000003
[ 989.932781]
[ 989.936059] INFO: task python3:5617 is blocked on a mutex likely owned by task python3:3562.
[ 989.946334] task:python3 state:D stack:25336 pid:3562 tgid:3562 ppid:3561 task_flags:0x400100 flags:0x00080802
[ 989.959538] Call Trace:
[ 989.963075]
[ 989.966226] __schedule+0xae3/0x14d0
[ 989.971038] ? io_schedule_timeout+0x140/0x140
[ 989.976822] ? find_held_lock+0x2b/0x80
[ 989.981924] ? __try_to_del_timer_sync+0x155/0x1f0
[ 989.988097] schedule+0xd4/0x260
[ 989.992514] schedule_timeout+0x13f/0x290
[ 989.997811] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 990.004182] ? trace_contention_end+0xb3/0x1a0
[ 990.009967] ? hrtimers_cpu_dying+0x260/0x260
[ 990.015657] ? ww_mutex_lock+0x160/0x160
[ 990.020857] msleep+0x1b/0x30
[ 990.024982] napi_set_threaded+0x3a5/0x440
[ 990.030381] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.035877] netif_set_threaded+0x257/0x400
[ 990.041370] threaded_store+0x1b8/0x220
[ 990.046473] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.051772] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.057069] ? sysfs_kf_write+0x65/0x160
[ 990.062262] ? sysfs_file_ops+0x140/0x140
[ 990.067556] kernfs_fop_write_iter+0x335/0x510
[ 990.073344] ? kernfs_ops+0xf0/0xf0
[ 990.078058] new_sync_write+0x333/0x750
[ 990.083160] ? __lock_acquire+0x508/0xc10
[ 990.088458] ? new_sync_read+0x740/0x740
[ 990.093650] ? lock_acquire.part.0+0xbc/0x260
[ 990.099335] ? ksys_write+0x116/0x250
[ 990.104246] vfs_write+0x657/0x9b0
[ 990.108860] ksys_write+0x116/0x250
[ 990.113572] ? __ia32_sys_read+0xc0/0xc0
[ 990.118774] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.124363] ? rcu_is_watching+0x15/0xd0
[ 990.129566] do_syscall_64+0xf3/0x5e0
[ 990.134474] ? trace_hardirqs_off+0xd/0x30
[ 990.139869] ? exc_page_fault+0xda/0xf0
[ 990.144969] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.151440] RIP: 0033:0x7fae7728422e
[ 990.156249] RSP: 002b:00007ffe2809d910 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.165548] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae7728422e
[ 990.174356] RDX: 0000000000000001 RSI: 00005617d4909f40 RDI: 0000000000000007
[ 990.183167] RBP: 00007ffe2809d920 R08: 0000000000000000 R09: 0000000000000000
[ 990.191976] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae770fc390
[ 990.200784] R13: 00005617d4909f40 R14: 0000000000000007 R15: 0000000000000003
[ 990.209597]
[ 990.212857] \x0aShowing all locks held in the system:
[ 990.221393] 1 lock held by khungtaskd/185:
[ 990.226788] #0: ffffffff8becdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.238263] 4 locks held by python3/3562:
[ 990.243558] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.253451] #1: ff1100016e866e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.264414] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.275574] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.285671] 4 locks held by python3/5617:
[ 990.290967] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.300855] #1: ff110001da524a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.311810] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.322970] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.333061]
[ 990.335526] =============================================\x0a
[ 1112.536771] INFO: task python3:5617 blocked for more than 245 seconds.
[ 1112.544921] Not tainted 7.0.0-rc6-elmk-g80a4209001ba #1
[ 1112.552172] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.561763] task:python3 state:D stack:25336 pid:5617 tgid:5617 ppid:5616 task_flags:0x400100 flags:0x00080000
[ 1112.574966] Call Trace:
[ 1112.578506]
[ 1112.581653] __schedule+0xae3/0x14d0
[ 1112.586467] ? io_schedule_timeout+0x140/0x140
[ 1112.592253] ? find_held_lock+0x2b/0x80
[ 1112.597358] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.603243] schedule+0xd4/0x260
[ 1112.607659] schedule_preempt_disabled+0x18/0x30
[ 1112.613635] __mutex_lock+0x6b2/0x1ee0
[ 1112.618640] ? rcu_is_watching+0x15/0xd0
[ 1112.623837] ? threaded_store+0x132/0x220
[ 1112.629135] ? ww_mutex_lock+0x160/0x160
[ 1112.634334] ? selinux_inode_free_security+0x330/0x330
[ 1112.640902] ? cap_capable+0x1ed/0x3f0
[ 1112.645911] ? threaded_store+0x132/0x220
[ 1112.651204] threaded_store+0x132/0x220
[ 1112.656304] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.661603] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.666904] ? sysfs_kf_write+0x65/0x160
[ 1112.672104] ? sysfs_file_ops+0x140/0x140
[ 1112.677390] kernfs_fop_write_iter+0x335/0x510
[ 1112.683173] ? kernfs_ops+0xf0/0xf0
[ 1112.687883] new_sync_write+0x333/0x750
[ 1112.692983] ? __lock_acquire+0x508/0xc10
[ 1112.698278] ? new_sync_read+0x740/0x740
[ 1112.703475] ? lock_acquire.part.0+0xbc/0x260
[ 1112.709161] ? ksys_write+0x116/0x250
[ 1112.714070] vfs_write+0x657/0x9b0
[ 1112.718683] ksys_write+0x116/0x250
[ 1112.723391] ? __ia32_sys_read+0xc0/0xc0
[ 1112.728588] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.734177] ? rcu_is_watching+0x15/0xd0
[ 1112.739374] do_syscall_64+0xf3/0x5e0
[ 1112.744279] ? irq_exit_rcu+0x39/0xf0
[ 1112.749184] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.755649] RIP: 0033:0x7f6887b1b22e
[ 1112.760456] RSP: 002b:00007fffdba8f830 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.769753] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6887b1b22e
[ 1112.778560] RDX: 0000000000000001 RSI: 0000558f9b964f40 RDI: 0000000000000007
[ 1112.787366] RBP: 00007fffdba8f840 R08: 0000000000000000 R09: 0000000000000000
[ 1112.796170] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6887993390
[ 1112.804976] R13: 0000558f9b964f40 R14: 0000000000000007 R15: 0000000000000003
[ 1112.813790]
[ 1112.817074] INFO: task python3:5617 is blocked on a mutex likely owned by task python3:3562.
[ 1112.827346] task:python3 state:D stack:25336 pid:3562 tgid:3562 ppid:3561 task_flags:0x400100 flags:0x00080802
[ 1112.840545] Call Trace:
[ 1112.844083]
[ 1112.847231] __schedule+0xae3/0x14d0
[ 1112.852039] ? io_schedule_timeout+0x140/0x140
[ 1112.857820] ? find_held_lock+0x2b/0x80
[ 1112.862921] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.869101] schedule+0xd4/0x260
[ 1112.873520] schedule_timeout+0x13f/0x290
[ 1112.878817] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 1112.885184] ? trace_contention_end+0xb3/0x1a0
[ 1112.890966] ? hrtimers_cpu_dying+0x260/0x260
[ 1112.896656] ? ww_mutex_lock+0x160/0x160
[ 1112.901853] msleep+0x1b/0x30
[ 1112.905976] napi_set_threaded+0x3a5/0x440
[ 1112.911371] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.916864] netif_set_threaded+0x257/0x400
[ 1112.922355] threaded_store+0x1b8/0x220
[ 1112.927454] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.932749] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.938042] ? sysfs_kf_write+0x65/0x160
[ 1112.943238] ? sysfs_file_ops+0x140/0x140
[ 1112.948530] kernfs_fop_write_iter+0x335/0x510
[ 1112.954313] ? kernfs_ops+0xf0/0xf0
[ 1112.959026] new_sync_write+0x333/0x750
[ 1112.964126] ? __lock_acquire+0x508/0xc10
[ 1112.969422] ? new_sync_read+0x740/0x740
[ 1112.974620] ? lock_acquire.part.0+0xbc/0x260
[ 1112.980303] ? ksys_write+0x116/0x250
[ 1112.985212] vfs_write+0x657/0x9b0
[ 1112.989826] ksys_write+0x116/0x250
[ 1112.994535] ? __ia32_sys_read+0xc0/0xc0
[ 1112.999734] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.005322] ? rcu_is_watching+0x15/0xd0
[ 1113.010522] do_syscall_64+0xf3/0x5e0
[ 1113.015419] ? trace_hardirqs_off+0xd/0x30
[ 1113.020812] ? exc_page_fault+0xda/0xf0
[ 1113.025911] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.032378] RIP: 0033:0x7fae7728422e
[ 1113.037175] RSP: 002b:00007ffe2809d910 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.046473] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae7728422e
[ 1113.055279] RDX: 0000000000000001 RSI: 00005617d4909f40 RDI: 0000000000000007
[ 1113.064087] RBP: 00007ffe2809d920 R08: 0000000000000000 R09: 0000000000000000
[ 1113.072895] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae770fc390
[ 1113.081703] R13: 00005617d4909f40 R14: 0000000000000007 R15: 0000000000000003
[ 1113.090507]
[ 1113.093772] \x0aShowing all locks held in the system:
[ 1113.102308] 1 lock held by khungtaskd/185:
[ 1113.107701] #0: ffffffff8becdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.119171] 2 locks held by systemd-journal/798:
[ 1113.125149] #0: ff1100180eab72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x194/0x14d0
[ 1113.135035] #1: ffffffff8c0c8fe0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.147860] 4 locks held by python3/3562:
[ 1113.153152] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.163038] #1: ff1100016e866e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.174001] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.185158] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.195242] 4 locks held by python3/5617:
[ 1113.200536] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.210422] #1: ff110001da524a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.221383] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.232540] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.242631]
[ 1113.245097] =============================================\x0a
[ 1235.416948] INFO: task python3:5617 blocked for more than 368 seconds.
[ 1235.425096] Not tainted 7.0.0-rc6-elmk-g80a4209001ba #1
[ 1235.432348] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.441939] task:python3 state:D stack:25336 pid:5617 tgid:5617 ppid:5616 task_flags:0x400100 flags:0x00080002
[ 1235.455144] Call Trace:
[ 1235.458687]
[ 1235.461827] __schedule+0xae3/0x14d0
[ 1235.466644] ? io_schedule_timeout+0x140/0x140
[ 1235.472428] ? find_held_lock+0x2b/0x80
[ 1235.477534] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.483423] schedule+0xd4/0x260
[ 1235.487843] schedule_preempt_disabled+0x18/0x30
[ 1235.493821] __mutex_lock+0x6b2/0x1ee0
[ 1235.498827] ? rcu_is_watching+0x15/0xd0
[ 1235.504027] ? threaded_store+0x132/0x220
[ 1235.509329] ? ww_mutex_lock+0x160/0x160
[ 1235.514530] ? selinux_inode_free_security+0x330/0x330
[ 1235.521104] ? cap_capable+0x1ed/0x3f0
[ 1235.526113] ? threaded_store+0x132/0x220
[ 1235.531412] threaded_store+0x132/0x220
[ 1235.536516] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.541814] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.547115] ? sysfs_kf_write+0x65/0x160
[ 1235.552317] ? sysfs_file_ops+0x140/0x140
[ 1235.557613] kernfs_fop_write_iter+0x335/0x510
[ 1235.563399] ? kernfs_ops+0xf0/0xf0
[ 1235.568109] new_sync_write+0x333/0x750
[ 1235.573216] ? __lock_acquire+0x508/0xc10
[ 1235.578513] ? new_sync_read+0x740/0x740
[ 1235.583707] ? lock_acquire.part.0+0xbc/0x260
[ 1235.589395] ? ksys_write+0x116/0x250
[ 1235.594307] vfs_write+0x657/0x9b0
[ 1235.598924] ksys_write+0x116/0x250
[ 1235.603638] ? __ia32_sys_read+0xc0/0xc0
[ 1235.608837] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.614429] ? rcu_is_watching+0x15/0xd0
[ 1235.619629] do_syscall_64+0xf3/0x5e0
[ 1235.624536] ? irq_exit_rcu+0x39/0xf0
[ 1235.629436] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.635898] RIP: 0033:0x7f6887b1b22e
[ 1235.640708] RSP: 002b:00007fffdba8f830 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.650006] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6887b1b22e
[ 1235.658813] RDX: 0000000000000001 RSI: 0000558f9b964f40 RDI: 0000000000000007
[ 1235.667621] RBP: 00007fffdba8f840 R08: 0000000000000000 R09: 0000000000000000
[ 1235.676420] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6887993390
[ 1235.685228] R13: 0000558f9b964f40 R14: 0000000000000007 R15: 0000000000000003
[ 1235.694033]
[ 1235.697304] INFO: task python3:5617 is blocked on a mutex likely owned by task python3:3562.
[ 1235.707642] task:python3 state:D stack:25336 pid:3562 tgid:3562 ppid:3561 task_flags:0x400100 flags:0x00080802
[ 1235.720845] Call Trace:
[ 1235.724382]
[ 1235.727531] __schedule+0xae3/0x14d0
[ 1235.732343] ? io_schedule_timeout+0x140/0x140
[ 1235.738130] ? find_held_lock+0x2b/0x80
[ 1235.743234] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.749415] schedule+0xd4/0x260
[ 1235.753834] schedule_timeout+0x13f/0x290
[ 1235.759130] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 1235.765502] ? trace_contention_end+0xb3/0x1a0
[ 1235.771290] ? hrtimers_cpu_dying+0x260/0x260
[ 1235.776981] ? ww_mutex_lock+0x160/0x160
[ 1235.782179] msleep+0x1b/0x30
[ 1235.786306] napi_set_threaded+0x3a5/0x440
[ 1235.791694] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.797182] netif_set_threaded+0x257/0x400
[ 1235.802677] threaded_store+0x1b8/0x220
[ 1235.807781] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.813081] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.818379] ? sysfs_kf_write+0x65/0x160
[ 1235.823569] ? sysfs_file_ops+0x140/0x140
[ 1235.828867] kernfs_fop_write_iter+0x335/0x510
[ 1235.834653] ? kernfs_ops+0xf0/0xf0
[ 1235.839434] new_sync_write+0x333/0x750
[ 1235.844537] ? __lock_acquire+0x508/0xc10
[ 1235.849827] ? new_sync_read+0x740/0x740
[ 1235.855029] ? lock_acquire.part.0+0xbc/0x260
[ 1235.860717] ? ksys_write+0x116/0x250
[ 1235.865627] vfs_write+0x657/0x9b0
[ 1235.870244] ksys_write+0x116/0x250
[ 1235.874960] ? __ia32_sys_read+0xc0/0xc0
[ 1235.880159] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.885739] ? rcu_is_watching+0x15/0xd0
[ 1235.890942] do_syscall_64+0xf3/0x5e0
[ 1235.895849] ? trace_hardirqs_off+0xd/0x30
[ 1235.901244] ? exc_page_fault+0xda/0xf0
[ 1235.906344] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.912814] RIP: 0033:0x7fae7728422e
[ 1235.917621] RSP: 002b:00007ffe2809d910 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.926922] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae7728422e
[ 1235.935729] RDX: 0000000000000001 RSI: 00005617d4909f40 RDI: 0000000000000007
[ 1235.944538] RBP: 00007ffe2809d920 R08: 0000000000000000 R09: 0000000000000000
[ 1235.953347] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae770fc390
[ 1235.962153] R13: 00005617d4909f40 R14: 0000000000000007 R15: 0000000000000003
[ 1235.970966]
[ 1235.974232] \x0aShowing all locks held in the system:
[ 1235.982764] 1 lock held by khungtaskd/185:
[ 1235.988161] #0: ffffffff8becdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1235.999646] 4 locks held by python3/3562:
[ 1236.004942] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.014895] #1: ff1100016e866e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.025857] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.037004] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.047098] 4 locks held by python3/5617:
[ 1236.052395] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.062281] #1: ff110001da524a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.073242] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.084400] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.094489] 2 locks held by systemd/7083:
[ 1236.099787]
[ 1236.102258] =============================================\x0a
[ 1358.299104] INFO: task python3:5617 blocked for more than 491 seconds.
[ 1358.307253] Not tainted 7.0.0-rc6-elmk-g80a4209001ba #1
[ 1358.314503] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.324097] task:python3 state:D stack:25336 pid:5617 tgid:5617 ppid:5616 task_flags:0x400100 flags:0x00080002
[ 1358.337305] Call Trace:
[ 1358.340847]
[ 1358.343997] __schedule+0xae3/0x14d0
[ 1358.348812] ? io_schedule_timeout+0x140/0x140
[ 1358.354599] ? find_held_lock+0x2b/0x80
[ 1358.359705] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.365597] schedule+0xd4/0x260
[ 1358.370019] schedule_preempt_disabled+0x18/0x30
[ 1358.376000] __mutex_lock+0x6b2/0x1ee0
[ 1358.381007] ? rcu_is_watching+0x15/0xd0
[ 1358.386205] ? threaded_store+0x132/0x220
[ 1358.391507] ? ww_mutex_lock+0x160/0x160
[ 1358.396706] ? selinux_inode_free_security+0x330/0x330
[ 1358.403275] ? cap_capable+0x1ed/0x3f0
[ 1358.408284] ? threaded_store+0x132/0x220
[ 1358.413582] threaded_store+0x132/0x220
[ 1358.418676] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.423977] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.429276] ? sysfs_kf_write+0x65/0x160
[ 1358.434476] ? sysfs_file_ops+0x140/0x140
[ 1358.439761] kernfs_fop_write_iter+0x335/0x510
[ 1358.445550] ? kernfs_ops+0xf0/0xf0
[ 1358.450262] new_sync_write+0x333/0x750
[ 1358.455356] ? __lock_acquire+0x508/0xc10
[ 1358.460655] ? new_sync_read+0x740/0x740
[ 1358.465856] ? lock_acquire.part.0+0xbc/0x260
[ 1358.471541] ? ksys_write+0x116/0x250
[ 1358.476453] vfs_write+0x657/0x9b0
[ 1358.481071] ksys_write+0x116/0x250
[ 1358.485784] ? __ia32_sys_read+0xc0/0xc0
[ 1358.490985] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.496576] ? rcu_is_watching+0x15/0xd0
[ 1358.501777] do_syscall_64+0xf3/0x5e0
[ 1358.506686] ? irq_exit_rcu+0x39/0xf0
[ 1358.511594] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.518056] RIP: 0033:0x7f6887b1b22e
[ 1358.522865] RSP: 002b:00007fffdba8f830 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.532164] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6887b1b22e
[ 1358.540975] RDX: 0000000000000001 RSI: 0000558f9b964f40 RDI: 0000000000000007
[ 1358.549785] RBP: 00007fffdba8f840 R08: 0000000000000000 R09: 0000000000000000
[ 1358.558593] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6887993390
[ 1358.567403] R13: 0000558f9b964f40 R14: 0000000000000007 R15: 0000000000000003
[ 1358.576218]
[ 1358.579487] INFO: task python3:5617 is blocked on a mutex likely owned by task python3:3562.
[ 1358.589762] task:python3 state:D stack:25336 pid:3562 tgid:3562 ppid:3561 task_flags:0x400100 flags:0x00080802
[ 1358.602966] Call Trace:
[ 1358.606504]
[ 1358.609653] __schedule+0xae3/0x14d0
[ 1358.614463] ? io_schedule_timeout+0x140/0x140
[ 1358.620240] ? find_held_lock+0x2b/0x80
[ 1358.625334] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.631513] schedule+0xd4/0x260
[ 1358.635930] schedule_timeout+0x13f/0x290
[ 1358.641228] ? hrtimer_nanosleep_restart+0x1c0/0x1c0
[ 1358.647598] ? trace_contention_end+0xb3/0x1a0
[ 1358.653383] ? hrtimers_cpu_dying+0x260/0x260
[ 1358.659074] ? ww_mutex_lock+0x160/0x160
[ 1358.664275] msleep+0x1b/0x30
[ 1358.668399] napi_set_threaded+0x3a5/0x440
[ 1358.673797] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.679293] netif_set_threaded+0x257/0x400
[ 1358.684788] threaded_store+0x1b8/0x220
[ 1358.689890] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.695189] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.700484] ? sysfs_kf_write+0x65/0x160
[ 1358.705684] ? sysfs_file_ops+0x140/0x140
[ 1358.710971] kernfs_fop_write_iter+0x335/0x510
[ 1358.716755] ? kernfs_ops+0xf0/0xf0
[ 1358.721466] new_sync_write+0x333/0x750
[ 1358.726569] ? __lock_acquire+0x508/0xc10
[ 1358.731867] ? new_sync_read+0x740/0x740
[ 1358.737067] ? lock_acquire.part.0+0xbc/0x260
[ 1358.742752] ? ksys_write+0x116/0x250
[ 1358.747664] vfs_write+0x657/0x9b0
[ 1358.752279] ksys_write+0x116/0x250
[ 1358.756992] ? __ia32_sys_read+0xc0/0xc0
[ 1358.762193] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.767785] ? rcu_is_watching+0x15/0xd0
[ 1358.772987] do_syscall_64+0xf3/0x5e0
[ 1358.777895] ? trace_hardirqs_off+0xd/0x30
[ 1358.783290] ? exc_page_fault+0xda/0xf0
[ 1358.788393] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.794861] RIP: 0033:0x7fae7728422e
[ 1358.799672] RSP: 002b:00007ffe2809d910 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.808969] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fae7728422e
[ 1358.817779] RDX: 0000000000000001 RSI: 00005617d4909f40 RDI: 0000000000000007
[ 1358.826588] RBP: 00007ffe2809d920 R08: 0000000000000000 R09: 0000000000000000
[ 1358.835395] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fae770fc390
[ 1358.844195] R13: 00005617d4909f40 R14: 0000000000000007 R15: 0000000000000003
[ 1358.853008]
[ 1358.856279] \x0aShowing all locks held in the system:
[ 1358.864819] 1 lock held by khungtaskd/185:
[ 1358.870214] #0: ffffffff8becdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.881687] 4 locks held by python3/3562:
[ 1358.886982] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.896871] #1: ff1100016e866e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.907823] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.918983] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.929078] 4 locks held by python3/5617:
[ 1358.934374] #0: ff1100012d9193f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.944265] #1: ff110001da524a88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.955228] #2: ff110001221910f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.966389] #3: ff110001221f9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.976481]
[ 1358.978953] =============================================\x0a