[ 762.253226] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 762.256582] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 762.384734] kselftest: Running tests in drivers/net
[ 989.712024] INFO: task python3:5309 blocked for more than 122 seconds.
[ 989.720260] Not tainted 7.1.0-rc6-edax-g9fc193549614 #1
[ 989.727501] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.737795] task:python3 state:D stack:25672 pid:5309 tgid:5309 ppid:5308 task_flags:0x400100 flags:0x00080000
[ 989.751001] Call Trace:
[ 989.754543]
[ 989.757693] __schedule+0xcc3/0x19c0
[ 989.762509] ? io_schedule_timeout+0x140/0x140
[ 989.768292] ? find_held_lock+0x2b/0x80
[ 989.773392] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.779277] schedule+0xd8/0x260
[ 989.783696] schedule_preempt_disabled+0x18/0x30
[ 989.789676] __mutex_lock+0x6af/0x2010
[ 989.794680] ? threaded_store+0x132/0x220
[ 989.799981] ? ww_mutex_lock+0x160/0x160
[ 989.805182] ? selinux_inode_free_security+0x330/0x330
[ 989.811755] ? cap_capable+0x1d6/0x3e0
[ 989.816766] ? threaded_store+0x132/0x220
[ 989.822064] threaded_store+0x132/0x220
[ 989.827161] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.832459] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.837759] ? sysfs_kf_write+0x65/0x160
[ 989.842957] ? sysfs_file_ops+0x140/0x140
[ 989.848252] kernfs_fop_write_iter+0x335/0x510
[ 989.854037] ? kernfs_ops+0xf0/0xf0
[ 989.858742] new_sync_write+0x333/0x750
[ 989.863847] ? __lock_acquire+0x508/0xc10
[ 989.869136] ? new_sync_read+0x740/0x740
[ 989.874337] ? lock_acquire.part.0+0xbc/0x260
[ 989.880022] ? ksys_write+0x116/0x250
[ 989.884935] vfs_write+0x657/0x9b0
[ 989.889550] ksys_write+0x116/0x250
[ 989.894256] ? __ia32_sys_read+0xc0/0xc0
[ 989.899457] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.905046] ? rcu_is_watching+0x15/0xd0
[ 989.910248] do_syscall_64+0xf3/0x650
[ 989.915155] ? trace_hardirqs_off+0xd/0x30
[ 989.920544] ? exc_page_fault+0xda/0xf0
[ 989.925645] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.932112] RIP: 0033:0x7f56635ab64e
[ 989.936929] RSP: 002b:00007fffa25e5430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.946226] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f56635ab64e
[ 989.955034] RDX: 0000000000000001 RSI: 0000561b43d66420 RDI: 0000000000000007
[ 989.963841] RBP: 00007fffa25e5440 R08: 0000000000000000 R09: 0000000000000000
[ 989.972650] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5663424390
[ 989.981456] R13: 0000561b43d66420 R14: 0000000000000007 R15: 00007f56633b9120
[ 989.990270]
[ 989.993578] INFO: task python3:5309 is blocked on a mutex likely owned by task python3:3325.
[ 990.003850] task:python3 state:D stack:25672 pid:3325 tgid:3325 ppid:3324 task_flags:0x400100 flags:0x00080802
[ 990.017052] Call Trace:
[ 990.020592]
[ 990.023742] __schedule+0xcc3/0x19c0
[ 990.028554] ? io_schedule_timeout+0x140/0x140
[ 990.034339] ? find_held_lock+0x2b/0x80
[ 990.039436] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.045615] schedule+0xd8/0x260
[ 990.050033] schedule_timeout+0x13f/0x290
[ 990.055332] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.061703] ? trace_contention_end+0xb3/0x1a0
[ 990.067478] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.073167] ? ww_mutex_lock+0x160/0x160
[ 990.078369] msleep+0x1b/0x30
[ 990.082497] napi_set_threaded+0x3a5/0x440
[ 990.087892] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.093387] netif_set_threaded+0x257/0x400
[ 990.098882] threaded_store+0x1b8/0x220
[ 990.103985] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.109276] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.114572] ? sysfs_kf_write+0x65/0x160
[ 990.119773] ? sysfs_file_ops+0x140/0x140
[ 990.125071] kernfs_fop_write_iter+0x335/0x510
[ 990.130856] ? kernfs_ops+0xf0/0xf0
[ 990.135570] new_sync_write+0x333/0x750
[ 990.140671] ? __lock_acquire+0x508/0xc10
[ 990.145967] ? new_sync_read+0x740/0x740
[ 990.151166] ? lock_acquire.part.0+0xbc/0x260
[ 990.156852] ? ksys_write+0x116/0x250
[ 990.161763] vfs_write+0x657/0x9b0
[ 990.166377] ksys_write+0x116/0x250
[ 990.171092] ? __ia32_sys_read+0xc0/0xc0
[ 990.176291] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.181882] ? rcu_is_watching+0x15/0xd0
[ 990.187083] do_syscall_64+0xf3/0x650
[ 990.191988] ? trace_hardirqs_off+0xd/0x30
[ 990.197385] ? exc_page_fault+0xda/0xf0
[ 990.202488] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.208952] RIP: 0033:0x7f1134fbb64e
[ 990.213762] RSP: 002b:00007ffe94faf0d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.223059] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1134fbb64e
[ 990.231865] RDX: 0000000000000001 RSI: 000055b1ecd23420 RDI: 0000000000000007
[ 990.240679] RBP: 00007ffe94faf0e0 R08: 0000000000000000 R09: 0000000000000000
[ 990.249488] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f1134e34390
[ 990.258297] R13: 000055b1ecd23420 R14: 0000000000000007 R15: 00007f1134dc9120
[ 990.267113]
[ 990.270370] \x0aShowing all locks held in the system:
[ 990.278911] 1 lock held by khungtaskd/185:
[ 990.284304] #0: ffffffff892fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.295781] 4 locks held by systemd-journal/805:
[ 990.301762] #0: ff1100180dab74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.311650] #1: ffffffff89502ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.324467] #2: ff1100019eba0b20 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.336018] #3: ff1100019f3256d8 (&xa->xa_lock#5){-.-.}-{3:3}, at: __folio_mark_dirty+0x9d/0x2c0
[ 990.346795] 4 locks held by python3/3325:
[ 990.352091] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.361978] #1: ff1100013d76f480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.372940] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.384100] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.394201] 4 locks held by python3/5309:
[ 990.399498] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.409385] #1: ff11000182577480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.420346] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.431500] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.441588]
[ 990.444056] =============================================\x0a
[ 1112.593516] INFO: task python3:5309 blocked for more than 245 seconds.
[ 1112.601666] Not tainted 7.1.0-rc6-edax-g9fc193549614 #1
[ 1112.608914] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.618501] task:python3 state:D stack:25672 pid:5309 tgid:5309 ppid:5308 task_flags:0x400100 flags:0x00080000
[ 1112.631705] Call Trace:
[ 1112.635245]
[ 1112.638393] __schedule+0xcc3/0x19c0
[ 1112.643207] ? io_schedule_timeout+0x140/0x140
[ 1112.648984] ? find_held_lock+0x2b/0x80
[ 1112.654086] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.659973] schedule+0xd8/0x260
[ 1112.664390] schedule_preempt_disabled+0x18/0x30
[ 1112.670368] __mutex_lock+0x6af/0x2010
[ 1112.675371] ? threaded_store+0x132/0x220
[ 1112.680672] ? ww_mutex_lock+0x160/0x160
[ 1112.685868] ? selinux_inode_free_security+0x330/0x330
[ 1112.692435] ? cap_capable+0x1d6/0x3e0
[ 1112.697441] ? threaded_store+0x132/0x220
[ 1112.702736] threaded_store+0x132/0x220
[ 1112.707838] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.713135] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.718434] ? sysfs_kf_write+0x65/0x160
[ 1112.723624] ? sysfs_file_ops+0x140/0x140
[ 1112.728916] kernfs_fop_write_iter+0x335/0x510
[ 1112.734699] ? kernfs_ops+0xf0/0xf0
[ 1112.739408] new_sync_write+0x333/0x750
[ 1112.744509] ? __lock_acquire+0x508/0xc10
[ 1112.749805] ? new_sync_read+0x740/0x740
[ 1112.755005] ? lock_acquire.part.0+0xbc/0x260
[ 1112.760691] ? ksys_write+0x116/0x250
[ 1112.765602] vfs_write+0x657/0x9b0
[ 1112.770216] ksys_write+0x116/0x250
[ 1112.774929] ? __ia32_sys_read+0xc0/0xc0
[ 1112.780128] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.785715] ? rcu_is_watching+0x15/0xd0
[ 1112.790918] do_syscall_64+0xf3/0x650
[ 1112.795823] ? trace_hardirqs_off+0xd/0x30
[ 1112.801215] ? exc_page_fault+0xda/0xf0
[ 1112.806318] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.812784] RIP: 0033:0x7f56635ab64e
[ 1112.817590] RSP: 002b:00007fffa25e5430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.826885] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f56635ab64e
[ 1112.835691] RDX: 0000000000000001 RSI: 0000561b43d66420 RDI: 0000000000000007
[ 1112.844496] RBP: 00007fffa25e5440 R08: 0000000000000000 R09: 0000000000000000
[ 1112.853302] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5663424390
[ 1112.862109] R13: 0000561b43d66420 R14: 0000000000000007 R15: 00007f56633b9120
[ 1112.870920]
[ 1112.874216] INFO: task python3:5309 is blocked on a mutex likely owned by task python3:3325.
[ 1112.884489] task:python3 state:D stack:25672 pid:3325 tgid:3325 ppid:3324 task_flags:0x400100 flags:0x00080802
[ 1112.897692] Call Trace:
[ 1112.901230]
[ 1112.904378] __schedule+0xcc3/0x19c0
[ 1112.909187] ? io_schedule_timeout+0x140/0x140
[ 1112.914969] ? find_held_lock+0x2b/0x80
[ 1112.920070] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.926249] schedule+0xd8/0x260
[ 1112.930667] schedule_timeout+0x13f/0x290
[ 1112.935962] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.942332] ? trace_contention_end+0xb3/0x1a0
[ 1112.948107] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.953795] ? ww_mutex_lock+0x160/0x160
[ 1112.958985] msleep+0x1b/0x30
[ 1112.963108] napi_set_threaded+0x3a5/0x440
[ 1112.968503] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.973994] netif_set_threaded+0x257/0x400
[ 1112.979486] threaded_store+0x1b8/0x220
[ 1112.984585] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.989881] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.995179] ? sysfs_kf_write+0x65/0x160
[ 1113.000375] ? sysfs_file_ops+0x140/0x140
[ 1113.005669] kernfs_fop_write_iter+0x335/0x510
[ 1113.011454] ? kernfs_ops+0xf0/0xf0
[ 1113.016162] new_sync_write+0x333/0x750
[ 1113.021262] ? __lock_acquire+0x508/0xc10
[ 1113.026558] ? new_sync_read+0x740/0x740
[ 1113.031758] ? lock_acquire.part.0+0xbc/0x260
[ 1113.037433] ? ksys_write+0x116/0x250
[ 1113.042340] vfs_write+0x657/0x9b0
[ 1113.046948] ksys_write+0x116/0x250
[ 1113.051659] ? __ia32_sys_read+0xc0/0xc0
[ 1113.056859] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.062447] ? rcu_is_watching+0x15/0xd0
[ 1113.067645] do_syscall_64+0xf3/0x650
[ 1113.072552] ? trace_hardirqs_off+0xd/0x30
[ 1113.077935] ? exc_page_fault+0xda/0xf0
[ 1113.083035] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.089502] RIP: 0033:0x7f1134fbb64e
[ 1113.094300] RSP: 002b:00007ffe94faf0d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.103598] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1134fbb64e
[ 1113.112406] RDX: 0000000000000001 RSI: 000055b1ecd23420 RDI: 0000000000000007
[ 1113.121211] RBP: 00007ffe94faf0e0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.130021] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f1134e34390
[ 1113.138827] R13: 000055b1ecd23420 R14: 0000000000000007 R15: 00007f1134dc9120
[ 1113.147640]
[ 1113.150904] \x0aShowing all locks held in the system:
[ 1113.159441] 1 lock held by khungtaskd/185:
[ 1113.164831] #0: ffffffff892fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.176316] 4 locks held by python3/3325:
[ 1113.181611] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.191498] #1: ff1100013d76f480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.202459] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.213615] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.223704] 4 locks held by python3/5309:
[ 1113.228999] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.238887] #1: ff11000182577480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.249848] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.261006] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.271098]
[ 1113.273563] =============================================\x0a
[ 1235.475682] INFO: task python3:5309 blocked for more than 368 seconds.
[ 1235.483832] Not tainted 7.1.0-rc6-edax-g9fc193549614 #1
[ 1235.491083] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.500671] task:python3 state:D stack:25672 pid:5309 tgid:5309 ppid:5308 task_flags:0x400100 flags:0x00080002
[ 1235.513874] Call Trace:
[ 1235.517414]
[ 1235.520561] __schedule+0xcc3/0x19c0
[ 1235.525375] ? io_schedule_timeout+0x140/0x140
[ 1235.531157] ? find_held_lock+0x2b/0x80
[ 1235.536260] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.542144] schedule+0xd8/0x260
[ 1235.546561] schedule_preempt_disabled+0x18/0x30
[ 1235.552538] __mutex_lock+0x6af/0x2010
[ 1235.557541] ? threaded_store+0x132/0x220
[ 1235.562842] ? ww_mutex_lock+0x160/0x160
[ 1235.568041] ? selinux_inode_free_security+0x330/0x330
[ 1235.574608] ? cap_capable+0x1d6/0x3e0
[ 1235.579613] ? threaded_store+0x132/0x220
[ 1235.584907] threaded_store+0x132/0x220
[ 1235.590010] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.595305] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.600605] ? sysfs_kf_write+0x65/0x160
[ 1235.605803] ? sysfs_file_ops+0x140/0x140
[ 1235.611096] kernfs_fop_write_iter+0x335/0x510
[ 1235.616879] ? kernfs_ops+0xf0/0xf0
[ 1235.621591] new_sync_write+0x333/0x750
[ 1235.626692] ? __lock_acquire+0x508/0xc10
[ 1235.631988] ? new_sync_read+0x740/0x740
[ 1235.637182] ? lock_acquire.part.0+0xbc/0x260
[ 1235.642868] ? ksys_write+0x116/0x250
[ 1235.647778] vfs_write+0x657/0x9b0
[ 1235.652392] ksys_write+0x116/0x250
[ 1235.657105] ? __ia32_sys_read+0xc0/0xc0
[ 1235.662303] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.667889] ? rcu_is_watching+0x15/0xd0
[ 1235.673091] do_syscall_64+0xf3/0x650
[ 1235.677995] ? trace_hardirqs_off+0xd/0x30
[ 1235.683387] ? exc_page_fault+0xda/0xf0
[ 1235.688488] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.694954] RIP: 0033:0x7f56635ab64e
[ 1235.699762] RSP: 002b:00007fffa25e5430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.709057] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f56635ab64e
[ 1235.717864] RDX: 0000000000000001 RSI: 0000561b43d66420 RDI: 0000000000000007
[ 1235.726670] RBP: 00007fffa25e5440 R08: 0000000000000000 R09: 0000000000000000
[ 1235.735478] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5663424390
[ 1235.744284] R13: 0000561b43d66420 R14: 0000000000000007 R15: 00007f56633b9120
[ 1235.753094]
[ 1235.756377] INFO: task python3:5309 is blocked on a mutex likely owned by task python3:3325.
[ 1235.766649] task:python3 state:D stack:25672 pid:3325 tgid:3325 ppid:3324 task_flags:0x400100 flags:0x00080802
[ 1235.779850] Call Trace:
[ 1235.783387]
[ 1235.786538] __schedule+0xcc3/0x19c0
[ 1235.791347] ? io_schedule_timeout+0x140/0x140
[ 1235.797129] ? find_held_lock+0x2b/0x80
[ 1235.802229] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.808407] schedule+0xd8/0x260
[ 1235.812822] schedule_timeout+0x13f/0x290
[ 1235.818120] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.824487] ? trace_contention_end+0xb3/0x1a0
[ 1235.830271] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.835959] ? ww_mutex_lock+0x160/0x160
[ 1235.841147] msleep+0x1b/0x30
[ 1235.845271] napi_set_threaded+0x3a5/0x440
[ 1235.850667] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.856154] netif_set_threaded+0x257/0x400
[ 1235.861647] threaded_store+0x1b8/0x220
[ 1235.866746] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.872043] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.877339] ? sysfs_kf_write+0x65/0x160
[ 1235.882538] ? sysfs_file_ops+0x140/0x140
[ 1235.887833] kernfs_fop_write_iter+0x335/0x510
[ 1235.893617] ? kernfs_ops+0xf0/0xf0
[ 1235.898327] new_sync_write+0x333/0x750
[ 1235.903426] ? __lock_acquire+0x508/0xc10
[ 1235.908722] ? new_sync_read+0x740/0x740
[ 1235.913921] ? lock_acquire.part.0+0xbc/0x260
[ 1235.919605] ? ksys_write+0x116/0x250
[ 1235.924512] vfs_write+0x657/0x9b0
[ 1235.929119] ksys_write+0x116/0x250
[ 1235.933820] ? __ia32_sys_read+0xc0/0xc0
[ 1235.939018] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.944607] ? rcu_is_watching+0x15/0xd0
[ 1235.949804] do_syscall_64+0xf3/0x650
[ 1235.954708] ? trace_hardirqs_off+0xd/0x30
[ 1235.960102] ? exc_page_fault+0xda/0xf0
[ 1235.965202] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.971668] RIP: 0033:0x7f1134fbb64e
[ 1235.976476] RSP: 002b:00007ffe94faf0d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.985770] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1134fbb64e
[ 1235.994575] RDX: 0000000000000001 RSI: 000055b1ecd23420 RDI: 0000000000000007
[ 1236.003383] RBP: 00007ffe94faf0e0 R08: 0000000000000000 R09: 0000000000000000
[ 1236.012188] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f1134e34390
[ 1236.020995] R13: 000055b1ecd23420 R14: 0000000000000007 R15: 00007f1134dc9120
[ 1236.029807]
[ 1236.033069] \x0aShowing all locks held in the system:
[ 1236.041604] 1 lock held by khungtaskd/185:
[ 1236.046994] #0: ffffffff892fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.058474] 4 locks held by python3/3325:
[ 1236.063770] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.073660] #1: ff1100013d76f480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.084622] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.095779] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.105870] 4 locks held by python3/5309:
[ 1236.111165] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.121051] #1: ff11000182577480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.132011] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.143167] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.153255]
[ 1236.155720] =============================================\x0a
[ 1358.356809] INFO: task python3:5309 blocked for more than 491 seconds.
[ 1358.364957] Not tainted 7.1.0-rc6-edax-g9fc193549614 #1
[ 1358.372200] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.381788] task:python3 state:D stack:25672 pid:5309 tgid:5309 ppid:5308 task_flags:0x400100 flags:0x00080002
[ 1358.394995] Call Trace:
[ 1358.398535]
[ 1358.401683] __schedule+0xcc3/0x19c0
[ 1358.406496] ? io_schedule_timeout+0x140/0x140
[ 1358.412283] ? find_held_lock+0x2b/0x80
[ 1358.417388] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.423271] schedule+0xd8/0x260
[ 1358.427689] schedule_preempt_disabled+0x18/0x30
[ 1358.433666] __mutex_lock+0x6af/0x2010
[ 1358.438669] ? threaded_store+0x132/0x220
[ 1358.443967] ? ww_mutex_lock+0x160/0x160
[ 1358.449166] ? selinux_inode_free_security+0x330/0x330
[ 1358.455737] ? cap_capable+0x1d6/0x3e0
[ 1358.460747] ? threaded_store+0x132/0x220
[ 1358.466043] threaded_store+0x132/0x220
[ 1358.471143] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.476442] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.481739] ? sysfs_kf_write+0x65/0x160
[ 1358.486940] ? sysfs_file_ops+0x140/0x140
[ 1358.492236] kernfs_fop_write_iter+0x335/0x510
[ 1358.498020] ? kernfs_ops+0xf0/0xf0
[ 1358.502731] new_sync_write+0x333/0x750
[ 1358.507833] ? __lock_acquire+0x508/0xc10
[ 1358.513127] ? new_sync_read+0x740/0x740
[ 1358.518325] ? lock_acquire.part.0+0xbc/0x260
[ 1358.524011] ? ksys_write+0x116/0x250
[ 1358.528911] vfs_write+0x657/0x9b0
[ 1358.533523] ksys_write+0x116/0x250
[ 1358.538232] ? __ia32_sys_read+0xc0/0xc0
[ 1358.543428] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.549018] ? rcu_is_watching+0x15/0xd0
[ 1358.554218] do_syscall_64+0xf3/0x650
[ 1358.559114] ? trace_hardirqs_off+0xd/0x30
[ 1358.564506] ? exc_page_fault+0xda/0xf0
[ 1358.569608] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.576073] RIP: 0033:0x7f56635ab64e
[ 1358.580879] RSP: 002b:00007fffa25e5430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.590174] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f56635ab64e
[ 1358.598980] RDX: 0000000000000001 RSI: 0000561b43d66420 RDI: 0000000000000007
[ 1358.607787] RBP: 00007fffa25e5440 R08: 0000000000000000 R09: 0000000000000000
[ 1358.616593] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5663424390
[ 1358.625399] R13: 0000561b43d66420 R14: 0000000000000007 R15: 00007f56633b9120
[ 1358.634210]
[ 1358.637517] INFO: task python3:5309 is blocked on a mutex likely owned by task python3:3325.
[ 1358.647789] task:python3 state:D stack:25672 pid:3325 tgid:3325 ppid:3324 task_flags:0x400100 flags:0x00080802
[ 1358.660990] Call Trace:
[ 1358.664522]
[ 1358.667669] __schedule+0xcc3/0x19c0
[ 1358.672479] ? io_schedule_timeout+0x140/0x140
[ 1358.678260] ? find_held_lock+0x2b/0x80
[ 1358.683363] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.689542] schedule+0xd8/0x260
[ 1358.693961] schedule_timeout+0x13f/0x290
[ 1358.699258] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.705616] ? trace_contention_end+0xb3/0x1a0
[ 1358.711399] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.717087] ? ww_mutex_lock+0x160/0x160
[ 1358.722275] msleep+0x1b/0x30
[ 1358.726398] napi_set_threaded+0x3a5/0x440
[ 1358.731794] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.737280] netif_set_threaded+0x257/0x400
[ 1358.742771] threaded_store+0x1b8/0x220
[ 1358.747865] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.753161] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.758458] ? sysfs_kf_write+0x65/0x160
[ 1358.763658] ? sysfs_file_ops+0x140/0x140
[ 1358.768943] kernfs_fop_write_iter+0x335/0x510
[ 1358.774728] ? kernfs_ops+0xf0/0xf0
[ 1358.779436] new_sync_write+0x333/0x750
[ 1358.784538] ? __lock_acquire+0x508/0xc10
[ 1358.789835] ? new_sync_read+0x740/0x740
[ 1358.795034] ? lock_acquire.part.0+0xbc/0x260
[ 1358.800718] ? ksys_write+0x116/0x250
[ 1358.805626] vfs_write+0x657/0x9b0
[ 1358.810240] ksys_write+0x116/0x250
[ 1358.814942] ? __ia32_sys_read+0xc0/0xc0
[ 1358.820138] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.825726] ? rcu_is_watching+0x15/0xd0
[ 1358.830923] do_syscall_64+0xf3/0x650
[ 1358.835830] ? trace_hardirqs_off+0xd/0x30
[ 1358.841223] ? exc_page_fault+0xda/0xf0
[ 1358.846324] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.852790] RIP: 0033:0x7f1134fbb64e
[ 1358.857597] RSP: 002b:00007ffe94faf0d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.866892] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1134fbb64e
[ 1358.875700] RDX: 0000000000000001 RSI: 000055b1ecd23420 RDI: 0000000000000007
[ 1358.884507] RBP: 00007ffe94faf0e0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.893314] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f1134e34390
[ 1358.902120] R13: 000055b1ecd23420 R14: 0000000000000007 R15: 00007f1134dc9120
[ 1358.910932]
[ 1358.914199] \x0aShowing all locks held in the system:
[ 1358.922739] 1 lock held by khungtaskd/185:
[ 1358.928129] #0: ffffffff892fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.939613] 4 locks held by python3/3325:
[ 1358.944901] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.954789] #1: ff1100013d76f480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.965750] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.976904] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.987004] 4 locks held by python3/5309:
[ 1358.992297] #0: ff1100012f8543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.002186] #1: ff11000182577480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.013137] #2: ff110001220eab50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.024293] #3: ff110001220d1d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.034387]
[ 1359.036853] =============================================\x0a