[ 782.464332] nipa-hw-worker: drivers/net:napi_threaded.py: timed out [ 782.466686] nipa-hw-worker: [5/45] Retrying drivers/net:napi_threaded.py [ 782.615805] kselftest: Running tests in drivers/net [ 989.717258] INFO: task python3:5758 blocked for more than 122 seconds. [ 989.725450] Not tainted 7.1.0-rc1-lgcj-ga14981f4a957 #1 [ 989.732983] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.742575] task:python3 state:D stack:25672 pid:5758 tgid:5758 ppid:5757 task_flags:0x400100 flags:0x00080000 [ 989.755783] Call Trace: [ 989.759321] [ 989.762473] __schedule+0xcba/0x1940 [ 989.767290] ? io_schedule_timeout+0x140/0x140 [ 989.773076] ? find_held_lock+0x2b/0x80 [ 989.778187] ? __lock_release.isra.0+0x6b/0x1a0 [ 989.784066] schedule+0xd4/0x260 [ 989.788486] schedule_preempt_disabled+0x18/0x30 [ 989.794457] __mutex_lock+0x6a3/0x2000 [ 989.799462] ? threaded_store+0x132/0x220 [ 989.804763] ? ww_mutex_lock+0x160/0x160 [ 989.809961] ? selinux_inode_free_security+0x330/0x330 [ 989.816533] ? cap_capable+0x1d6/0x3e0 [ 989.821544] ? threaded_store+0x132/0x220 [ 989.826841] threaded_store+0x132/0x220 [ 989.831944] ? xps_rxqs_store+0x2e0/0x2e0 [ 989.837240] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.842532] ? sysfs_kf_write+0x65/0x160 [ 989.847731] ? sysfs_file_ops+0x140/0x140 [ 989.853027] kernfs_fop_write_iter+0x335/0x510 [ 989.858813] ? kernfs_ops+0xf0/0xf0 [ 989.863526] new_sync_write+0x333/0x750 [ 989.868630] ? __lock_acquire+0x508/0xc10 [ 989.873929] ? new_sync_read+0x740/0x740 [ 989.879129] ? lock_acquire.part.0+0xbc/0x260 [ 989.884815] ? ksys_write+0x116/0x250 [ 989.889727] vfs_write+0x657/0x9b0 [ 989.894342] ksys_write+0x116/0x250 [ 989.899044] ? __ia32_sys_read+0xc0/0xc0 [ 989.904245] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.909834] ? rcu_is_watching+0x15/0xd0 [ 989.915037] do_syscall_64+0xf3/0x650 [ 989.919944] ? irq_exit_rcu+0x1a/0x30 [ 989.924850] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.931318] RIP: 0033:0x7ff6d921922e [ 989.936128] RSP: 002b:00007ffc02261460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.945416] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff6d921922e [ 989.954226] RDX: 0000000000000001 RSI: 0000558b7c395e10 RDI: 0000000000000007 [ 989.963034] RBP: 00007ffc02261470 R08: 0000000000000000 R09: 0000000000000000 [ 989.971842] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff6d9091390 [ 989.980652] R13: 0000558b7c395e10 R14: 0000000000000007 R15: 0000000000000003 [ 989.989466] [ 989.992761] INFO: task python3:5758 is blocked on a mutex likely owned by task python3:3668. [ 990.003036] task:python3 state:D stack:25672 pid:3668 tgid:3668 ppid:3667 task_flags:0x400100 flags:0x00080802 [ 990.016228] Call Trace: [ 990.019770] [ 990.022919] __schedule+0xcba/0x1940 [ 990.027728] ? io_schedule_timeout+0x140/0x140 [ 990.033514] ? find_held_lock+0x2b/0x80 [ 990.038618] ? __try_to_del_timer_sync+0x155/0x1f0 [ 990.044799] schedule+0xd4/0x260 [ 990.049220] schedule_timeout+0x13f/0x290 [ 990.054516] ? hrtimer_nanosleep_restart+0x270/0x270 [ 990.060885] ? trace_contention_end+0xb3/0x1a0 [ 990.066669] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 990.072360] ? ww_mutex_lock+0x160/0x160 [ 990.077567] msleep+0x1b/0x30 [ 990.081691] napi_set_threaded+0x3a5/0x440 [ 990.087092] ? napi_resume_irqs+0x1a0/0x1a0 [ 990.092588] netif_set_threaded+0x257/0x400 [ 990.098081] threaded_store+0x1b8/0x220 [ 990.103187] ? xps_rxqs_store+0x2e0/0x2e0 [ 990.108484] ? sysfs_file_kobj+0xbd/0x1e0 [ 990.113781] ? sysfs_kf_write+0x65/0x160 [ 990.118983] ? sysfs_file_ops+0x140/0x140 [ 990.124277] kernfs_fop_write_iter+0x335/0x510 [ 990.130063] ? kernfs_ops+0xf0/0xf0 [ 990.134777] new_sync_write+0x333/0x750 [ 990.139882] ? __lock_acquire+0x508/0xc10 [ 990.145179] ? new_sync_read+0x740/0x740 [ 990.150369] ? lock_acquire.part.0+0xbc/0x260 [ 990.156058] ? ksys_write+0x116/0x250 [ 990.160969] vfs_write+0x657/0x9b0 [ 990.165582] ksys_write+0x116/0x250 [ 990.170293] ? __ia32_sys_read+0xc0/0xc0 [ 990.175495] ? kernfs_fop_llseek+0x160/0x1f0 [ 990.181083] ? rcu_is_watching+0x15/0xd0 [ 990.186287] do_syscall_64+0xf3/0x650 [ 990.191195] ? irq_exit_rcu+0x1a/0x30 [ 990.196101] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 990.202569] RIP: 0033:0x7f7e9aa4122e [ 990.207371] RSP: 002b:00007fffaf867fa0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 990.216670] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e9aa4122e [ 990.225478] RDX: 0000000000000001 RSI: 000055dfac700e10 RDI: 0000000000000007 [ 990.234285] RBP: 00007fffaf867fb0 R08: 0000000000000000 R09: 0000000000000000 [ 990.243098] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e9a8b9390 [ 990.251904] R13: 000055dfac700e10 R14: 0000000000000007 R15: 0000000000000003 [ 990.260716] [ 990.263976] \x0aShowing all locks held in the system: [ 990.272515] 1 lock held by khungtaskd/185: [ 990.277908] #0: ffffffff918fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 990.289390] 4 locks held by python3/3668: [ 990.294684] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 990.304573] #1: ff11000166200880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 990.315538] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.326697] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.336797] 4 locks held by python3/5758: [ 990.342095] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 990.351976] #1: ff110001e7633280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 990.362940] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.374102] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.384193] [ 990.386658] =============================================\x0a [ 1079.829159] kworker/u129:14 (311) used greatest stack depth: 21456 bytes left [ 1112.597509] INFO: task python3:5758 blocked for more than 245 seconds. [ 1112.605658] Not tainted 7.1.0-rc1-lgcj-ga14981f4a957 #1 [ 1112.612906] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.622494] task:python3 state:D stack:25672 pid:5758 tgid:5758 ppid:5757 task_flags:0x400100 flags:0x00080000 [ 1112.635698] Call Trace: [ 1112.639237] [ 1112.642385] __schedule+0xcba/0x1940 [ 1112.647198] ? io_schedule_timeout+0x140/0x140 [ 1112.652985] ? find_held_lock+0x2b/0x80 [ 1112.658087] ? __lock_release.isra.0+0x6b/0x1a0 [ 1112.663971] schedule+0xd4/0x260 [ 1112.668387] schedule_preempt_disabled+0x18/0x30 [ 1112.674365] __mutex_lock+0x6a3/0x2000 [ 1112.679367] ? threaded_store+0x132/0x220 [ 1112.684665] ? ww_mutex_lock+0x160/0x160 [ 1112.689860] ? selinux_inode_free_security+0x330/0x330 [ 1112.696429] ? cap_capable+0x1d6/0x3e0 [ 1112.701434] ? threaded_store+0x132/0x220 [ 1112.706728] threaded_store+0x132/0x220 [ 1112.711829] ? xps_rxqs_store+0x2e0/0x2e0 [ 1112.717123] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.722421] ? sysfs_kf_write+0x65/0x160 [ 1112.727624] ? sysfs_file_ops+0x140/0x140 [ 1112.732916] kernfs_fop_write_iter+0x335/0x510 [ 1112.738699] ? kernfs_ops+0xf0/0xf0 [ 1112.743410] new_sync_write+0x333/0x750 [ 1112.748502] ? __lock_acquire+0x508/0xc10 [ 1112.753797] ? new_sync_read+0x740/0x740 [ 1112.758995] ? lock_acquire.part.0+0xbc/0x260 [ 1112.764682] ? ksys_write+0x116/0x250 [ 1112.769592] vfs_write+0x657/0x9b0 [ 1112.774205] ksys_write+0x116/0x250 [ 1112.778917] ? __ia32_sys_read+0xc0/0xc0 [ 1112.784106] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.789693] ? rcu_is_watching+0x15/0xd0 [ 1112.794894] do_syscall_64+0xf3/0x650 [ 1112.799799] ? irq_exit_rcu+0x1a/0x30 [ 1112.804703] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.811169] RIP: 0033:0x7ff6d921922e [ 1112.815980] RSP: 002b:00007ffc02261460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.825279] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff6d921922e [ 1112.834085] RDX: 0000000000000001 RSI: 0000558b7c395e10 RDI: 0000000000000007 [ 1112.842891] RBP: 00007ffc02261470 R08: 0000000000000000 R09: 0000000000000000 [ 1112.851698] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff6d9091390 [ 1112.860505] R13: 0000558b7c395e10 R14: 0000000000000007 R15: 0000000000000003 [ 1112.869316] [ 1112.872618] INFO: task python3:5758 is blocked on a mutex likely owned by task python3:3668. [ 1112.882894] task:python3 state:D stack:25672 pid:3668 tgid:3668 ppid:3667 task_flags:0x400100 flags:0x00080802 [ 1112.896097] Call Trace: [ 1112.899634] [ 1112.902782] __schedule+0xcba/0x1940 [ 1112.907590] ? io_schedule_timeout+0x140/0x140 [ 1112.913372] ? find_held_lock+0x2b/0x80 [ 1112.918473] ? __try_to_del_timer_sync+0x155/0x1f0 [ 1112.924645] schedule+0xd4/0x260 [ 1112.929059] schedule_timeout+0x13f/0x290 [ 1112.934356] ? hrtimer_nanosleep_restart+0x270/0x270 [ 1112.940722] ? trace_contention_end+0xb3/0x1a0 [ 1112.946505] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 1112.952194] ? ww_mutex_lock+0x160/0x160 [ 1112.957390] msleep+0x1b/0x30 [ 1112.961514] napi_set_threaded+0x3a5/0x440 [ 1112.966909] ? napi_resume_irqs+0x1a0/0x1a0 [ 1112.972404] netif_set_threaded+0x257/0x400 [ 1112.977896] threaded_store+0x1b8/0x220 [ 1112.982988] ? xps_rxqs_store+0x2e0/0x2e0 [ 1112.988283] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.993578] ? sysfs_kf_write+0x65/0x160 [ 1112.998775] ? sysfs_file_ops+0x140/0x140 [ 1113.004068] kernfs_fop_write_iter+0x335/0x510 [ 1113.009852] ? kernfs_ops+0xf0/0xf0 [ 1113.014560] new_sync_write+0x333/0x750 [ 1113.019660] ? __lock_acquire+0x508/0xc10 [ 1113.024956] ? new_sync_read+0x740/0x740 [ 1113.030156] ? lock_acquire.part.0+0xbc/0x260 [ 1113.035838] ? ksys_write+0x116/0x250 [ 1113.040749] vfs_write+0x657/0x9b0 [ 1113.045363] ksys_write+0x116/0x250 [ 1113.050074] ? __ia32_sys_read+0xc0/0xc0 [ 1113.055272] ? kernfs_fop_llseek+0x160/0x1f0 [ 1113.060859] ? rcu_is_watching+0x15/0xd0 [ 1113.066060] do_syscall_64+0xf3/0x650 [ 1113.070965] ? irq_exit_rcu+0x1a/0x30 [ 1113.075872] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1113.082329] RIP: 0033:0x7f7e9aa4122e [ 1113.087139] RSP: 002b:00007fffaf867fa0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1113.096433] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e9aa4122e [ 1113.105240] RDX: 0000000000000001 RSI: 000055dfac700e10 RDI: 0000000000000007 [ 1113.114047] RBP: 00007fffaf867fb0 R08: 0000000000000000 R09: 0000000000000000 [ 1113.122854] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e9a8b9390 [ 1113.131660] R13: 000055dfac700e10 R14: 0000000000000007 R15: 0000000000000003 [ 1113.140475] [ 1113.143738] \x0aShowing all locks held in the system: [ 1113.152282] 1 lock held by khungtaskd/185: [ 1113.157676] #0: ffffffff918fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1113.169163] 4 locks held by python3/3668: [ 1113.174456] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1113.184346] #1: ff11000166200880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1113.195310] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.206466] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.216567] 4 locks held by python3/5758: [ 1113.221861] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1113.231749] #1: ff110001e7633280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1113.242708] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.253863] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.263952] [ 1113.266417] =============================================\x0a [ 1235.478716] INFO: task python3:5758 blocked for more than 368 seconds. [ 1235.486864] Not tainted 7.1.0-rc1-lgcj-ga14981f4a957 #1 [ 1235.494105] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.503691] task:python3 state:D stack:25672 pid:5758 tgid:5758 ppid:5757 task_flags:0x400100 flags:0x00080002 [ 1235.516895] Call Trace: [ 1235.520435] [ 1235.523583] __schedule+0xcba/0x1940 [ 1235.528396] ? io_schedule_timeout+0x140/0x140 [ 1235.534183] ? find_held_lock+0x2b/0x80 [ 1235.539287] ? __lock_release.isra.0+0x6b/0x1a0 [ 1235.545172] schedule+0xd4/0x260 [ 1235.549588] schedule_preempt_disabled+0x18/0x30 [ 1235.555565] __mutex_lock+0x6a3/0x2000 [ 1235.560568] ? threaded_store+0x132/0x220 [ 1235.565866] ? ww_mutex_lock+0x160/0x160 [ 1235.571065] ? selinux_inode_free_security+0x330/0x330 [ 1235.577636] ? cap_capable+0x1d6/0x3e0 [ 1235.582643] ? threaded_store+0x132/0x220 [ 1235.587939] threaded_store+0x132/0x220 [ 1235.593041] ? xps_rxqs_store+0x2e0/0x2e0 [ 1235.598339] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.603636] ? sysfs_kf_write+0x65/0x160 [ 1235.608833] ? sysfs_file_ops+0x140/0x140 [ 1235.614125] kernfs_fop_write_iter+0x335/0x510 [ 1235.619908] ? kernfs_ops+0xf0/0xf0 [ 1235.624619] new_sync_write+0x333/0x750 [ 1235.629720] ? __lock_acquire+0x508/0xc10 [ 1235.635016] ? new_sync_read+0x740/0x740 [ 1235.640214] ? lock_acquire.part.0+0xbc/0x260 [ 1235.645899] ? ksys_write+0x116/0x250 [ 1235.650809] vfs_write+0x657/0x9b0 [ 1235.655423] ksys_write+0x116/0x250 [ 1235.660133] ? __ia32_sys_read+0xc0/0xc0 [ 1235.665331] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.670919] ? rcu_is_watching+0x15/0xd0 [ 1235.676109] do_syscall_64+0xf3/0x650 [ 1235.681014] ? irq_exit_rcu+0x1a/0x30 [ 1235.685922] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.692391] RIP: 0033:0x7ff6d921922e [ 1235.697205] RSP: 002b:00007ffc02261460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.706504] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff6d921922e [ 1235.715310] RDX: 0000000000000001 RSI: 0000558b7c395e10 RDI: 0000000000000007 [ 1235.724115] RBP: 00007ffc02261470 R08: 0000000000000000 R09: 0000000000000000 [ 1235.732923] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff6d9091390 [ 1235.741731] R13: 0000558b7c395e10 R14: 0000000000000007 R15: 0000000000000003 [ 1235.750543] [ 1235.753839] INFO: task python3:5758 is blocked on a mutex likely owned by task python3:3668. [ 1235.764110] task:python3 state:D stack:25672 pid:3668 tgid:3668 ppid:3667 task_flags:0x400100 flags:0x00080802 [ 1235.777310] Call Trace: [ 1235.780839] [ 1235.783987] __schedule+0xcba/0x1940 [ 1235.788795] ? io_schedule_timeout+0x140/0x140 [ 1235.794577] ? find_held_lock+0x2b/0x80 [ 1235.799678] ? __try_to_del_timer_sync+0x155/0x1f0 [ 1235.805858] schedule+0xd4/0x260 [ 1235.810272] schedule_timeout+0x13f/0x290 [ 1235.815570] ? hrtimer_nanosleep_restart+0x270/0x270 [ 1235.821928] ? trace_contention_end+0xb3/0x1a0 [ 1235.827711] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 1235.833399] ? ww_mutex_lock+0x160/0x160 [ 1235.838596] msleep+0x1b/0x30 [ 1235.842721] napi_set_threaded+0x3a5/0x440 [ 1235.848117] ? napi_resume_irqs+0x1a0/0x1a0 [ 1235.853612] netif_set_threaded+0x257/0x400 [ 1235.859105] threaded_store+0x1b8/0x220 [ 1235.864197] ? xps_rxqs_store+0x2e0/0x2e0 [ 1235.869493] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.874789] ? sysfs_kf_write+0x65/0x160 [ 1235.879987] ? sysfs_file_ops+0x140/0x140 [ 1235.885281] kernfs_fop_write_iter+0x335/0x510 [ 1235.891065] ? kernfs_ops+0xf0/0xf0 [ 1235.895774] new_sync_write+0x333/0x750 [ 1235.900875] ? __lock_acquire+0x508/0xc10 [ 1235.906173] ? new_sync_read+0x740/0x740 [ 1235.911372] ? lock_acquire.part.0+0xbc/0x260 [ 1235.917054] ? ksys_write+0x116/0x250 [ 1235.921965] vfs_write+0x657/0x9b0 [ 1235.926579] ksys_write+0x116/0x250 [ 1235.931290] ? __ia32_sys_read+0xc0/0xc0 [ 1235.936489] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.942077] ? rcu_is_watching+0x15/0xd0 [ 1235.947277] do_syscall_64+0xf3/0x650 [ 1235.952182] ? irq_exit_rcu+0x1a/0x30 [ 1235.957087] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.963553] RIP: 0033:0x7f7e9aa4122e [ 1235.968364] RSP: 002b:00007fffaf867fa0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.977657] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e9aa4122e [ 1235.986464] RDX: 0000000000000001 RSI: 000055dfac700e10 RDI: 0000000000000007 [ 1235.995270] RBP: 00007fffaf867fb0 R08: 0000000000000000 R09: 0000000000000000 [ 1236.004077] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e9a8b9390 [ 1236.012883] R13: 000055dfac700e10 R14: 0000000000000007 R15: 0000000000000003 [ 1236.021695] [ 1236.024960] \x0aShowing all locks held in the system: [ 1236.033499] 1 lock held by khungtaskd/185: [ 1236.038893] #0: ffffffff918fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1236.050372] 4 locks held by python3/3668: [ 1236.055665] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1236.065553] #1: ff11000166200880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1236.076518] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.087671] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.097768] 4 locks held by python3/5758: [ 1236.103063] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1236.112951] #1: ff110001e7633280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1236.123910] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.135067] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.145150] [ 1236.147616] =============================================\x0a [ 1358.360938] INFO: task python3:5758 blocked for more than 491 seconds. [ 1358.369079] Not tainted 7.1.0-rc1-lgcj-ga14981f4a957 #1 [ 1358.376327] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.385908] task:python3 state:D stack:25672 pid:5758 tgid:5758 ppid:5757 task_flags:0x400100 flags:0x00080002 [ 1358.399105] Call Trace: [ 1358.402644] [ 1358.405796] __schedule+0xcba/0x1940 [ 1358.410612] ? io_schedule_timeout+0x140/0x140 [ 1358.416398] ? find_held_lock+0x2b/0x80 [ 1358.421503] ? __lock_release.isra.0+0x6b/0x1a0 [ 1358.427392] schedule+0xd4/0x260 [ 1358.431811] schedule_preempt_disabled+0x18/0x30 [ 1358.437791] __mutex_lock+0x6a3/0x2000 [ 1358.442796] ? threaded_store+0x132/0x220 [ 1358.448097] ? ww_mutex_lock+0x160/0x160 [ 1358.453294] ? selinux_inode_free_security+0x330/0x330 [ 1358.459866] ? cap_capable+0x1d6/0x3e0 [ 1358.464873] ? threaded_store+0x132/0x220 [ 1358.470169] threaded_store+0x132/0x220 [ 1358.475272] ? xps_rxqs_store+0x2e0/0x2e0 [ 1358.480568] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.485860] ? sysfs_kf_write+0x65/0x160 [ 1358.491062] ? sysfs_file_ops+0x140/0x140 [ 1358.496358] kernfs_fop_write_iter+0x335/0x510 [ 1358.502142] ? kernfs_ops+0xf0/0xf0 [ 1358.506855] new_sync_write+0x333/0x750 [ 1358.511958] ? __lock_acquire+0x508/0xc10 [ 1358.517256] ? new_sync_read+0x740/0x740 [ 1358.522457] ? lock_acquire.part.0+0xbc/0x260 [ 1358.528142] ? ksys_write+0x116/0x250 [ 1358.533054] vfs_write+0x657/0x9b0 [ 1358.537669] ksys_write+0x116/0x250 [ 1358.542380] ? __ia32_sys_read+0xc0/0xc0 [ 1358.547580] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.553169] ? rcu_is_watching+0x15/0xd0 [ 1358.558372] do_syscall_64+0xf3/0x650 [ 1358.563278] ? irq_exit_rcu+0x1a/0x30 [ 1358.568185] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.574653] RIP: 0033:0x7ff6d921922e [ 1358.579465] RSP: 002b:00007ffc02261460 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.588762] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff6d921922e [ 1358.597561] RDX: 0000000000000001 RSI: 0000558b7c395e10 RDI: 0000000000000007 [ 1358.606359] RBP: 00007ffc02261470 R08: 0000000000000000 R09: 0000000000000000 [ 1358.615168] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff6d9091390 [ 1358.623976] R13: 0000558b7c395e10 R14: 0000000000000007 R15: 0000000000000003 [ 1358.632795] [ 1358.636067] INFO: task python3:5758 is blocked on a mutex likely owned by task python3:3668. [ 1358.646340] task:python3 state:D stack:25672 pid:3668 tgid:3668 ppid:3667 task_flags:0x400100 flags:0x00080802 [ 1358.659542] Call Trace: [ 1358.663081] [ 1358.666230] __schedule+0xcba/0x1940 [ 1358.671040] ? io_schedule_timeout+0x140/0x140 [ 1358.676823] ? find_held_lock+0x2b/0x80 [ 1358.681926] ? __try_to_del_timer_sync+0x155/0x1f0 [ 1358.688108] schedule+0xd4/0x260 [ 1358.692525] schedule_timeout+0x13f/0x290 [ 1358.697822] ? hrtimer_nanosleep_restart+0x270/0x270 [ 1358.704191] ? trace_contention_end+0xb3/0x1a0 [ 1358.709976] ? hrtimers_cpu_dying+0x3e0/0x3e0 [ 1358.715667] ? ww_mutex_lock+0x160/0x160 [ 1358.720864] msleep+0x1b/0x30 [ 1358.724990] napi_set_threaded+0x3a5/0x440 [ 1358.730387] ? napi_resume_irqs+0x1a0/0x1a0 [ 1358.735882] netif_set_threaded+0x257/0x400 [ 1358.741378] threaded_store+0x1b8/0x220 [ 1358.746480] ? xps_rxqs_store+0x2e0/0x2e0 [ 1358.751771] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.757069] ? sysfs_kf_write+0x65/0x160 [ 1358.762268] ? sysfs_file_ops+0x140/0x140 [ 1358.767567] kernfs_fop_write_iter+0x335/0x510 [ 1358.773351] ? kernfs_ops+0xf0/0xf0 [ 1358.778065] new_sync_write+0x333/0x750 [ 1358.783165] ? __lock_acquire+0x508/0xc10 [ 1358.788462] ? new_sync_read+0x740/0x740 [ 1358.793665] ? lock_acquire.part.0+0xbc/0x260 [ 1358.799353] ? ksys_write+0x116/0x250 [ 1358.804264] vfs_write+0x657/0x9b0 [ 1358.808878] ksys_write+0x116/0x250 [ 1358.813580] ? __ia32_sys_read+0xc0/0xc0 [ 1358.818787] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.824375] ? rcu_is_watching+0x15/0xd0 [ 1358.829575] do_syscall_64+0xf3/0x650 [ 1358.834472] ? irq_exit_rcu+0x1a/0x30 [ 1358.839380] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.845847] RIP: 0033:0x7f7e9aa4122e [ 1358.850645] RSP: 002b:00007fffaf867fa0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.859941] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7e9aa4122e [ 1358.868747] RDX: 0000000000000001 RSI: 000055dfac700e10 RDI: 0000000000000007 [ 1358.877557] RBP: 00007fffaf867fb0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.886364] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f7e9a8b9390 [ 1358.895163] R13: 000055dfac700e10 R14: 0000000000000007 R15: 0000000000000003 [ 1358.903978] [ 1358.907245] \x0aShowing all locks held in the system: [ 1358.915788] 1 lock held by khungtaskd/185: [ 1358.921181] #0: ffffffff918fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1358.932653] 3 locks held by systemd-journal/805: [ 1358.938631] #0: ff1100180dd372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940 [ 1358.948520] #1: ff110001a3e48378 (&mm->mmap_lock){++++}-{4:4}, at: fs_reclaim_acquire+0x7b/0x120 [ 1358.959289] #2: ff1100011be85150 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0 [ 1358.970850] 4 locks held by python3/3668: [ 1358.976146] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1358.986035] #1: ff11000166200880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1358.996998] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1359.008154] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1359.018249] 4 locks held by python3/5758: [ 1359.023545] #0: ff11000130be13e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250 [ 1359.033432] #1: ff110001e7633280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 [ 1359.044391] #2: ff11000123cbeb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1359.055545] #3: ff11000123d91d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1359.065635] [ 1359.068100] =============================================\x0a