[ 781.653395] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 781.655922] nipa-hw-worker: [4/40] Retrying drivers/net:napi_threaded.py
[ 781.793698] kselftest: Running tests in drivers/net
[ 989.714243] INFO: task python3:5440 blocked for more than 122 seconds.
[ 989.722473] Not tainted 7.0.0-rc6-qbvg-ga64f8f00fd9b #1
[ 989.729724] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.739314] task:python3 state:D stack:25312 pid:5440 tgid:5440 ppid:5439 task_flags:0x400100 flags:0x00080000
[ 989.752521] Call Trace:
[ 989.756063]
[ 989.759215] __schedule+0xad9/0x1490
[ 989.764035] ? io_schedule_timeout+0x130/0x130
[ 989.769820] ? find_held_lock+0x2b/0x80
[ 989.774927] ? __lock_release.isra.0+0x59/0x170
[ 989.780813] schedule+0xd4/0x260
[ 989.785231] schedule_preempt_disabled+0x18/0x30
[ 989.791210] __mutex_lock+0xd76/0x1f20
[ 989.796217] ? threaded_store+0x132/0x220
[ 989.801517] ? ww_mutex_lock+0x160/0x160
[ 989.806718] ? selinux_inode_free_security+0x310/0x310
[ 989.813290] ? cap_capable+0x1ed/0x3f0
[ 989.818299] ? sysfs_file_ops+0x140/0x140
[ 989.823600] ? threaded_store+0x132/0x220
[ 989.828896] threaded_store+0x132/0x220
[ 989.834000] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.839495] ? sysfs_file_ops+0x140/0x140
[ 989.844790] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.850088] ? sysfs_kf_write+0x65/0x160
[ 989.855277] kernfs_fop_write_iter+0x335/0x510
[ 989.861066] ? kernfs_ops+0xf0/0xf0
[ 989.865782] new_sync_write+0x308/0x6f0
[ 989.870886] ? new_sync_read+0x6f0/0x6f0
[ 989.876091] ? lock_acquire.part.0+0xbc/0x260
[ 989.881780] ? ksys_write+0xff/0x200
[ 989.886593] vfs_write+0x62a/0x980
[ 989.891209] ksys_write+0xff/0x200
[ 989.895824] ? __ia32_sys_read+0xc0/0xc0
[ 989.901014] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.906603] ? rcu_is_watching+0x15/0xd0
[ 989.911796] do_syscall_64+0xf3/0x5e0
[ 989.916702] ? exc_page_fault+0x9e/0xb0
[ 989.921806] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.928277] RIP: 0033:0x7f001f06ac5e
[ 989.933089] RSP: 002b:00007ffe03120960 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.942397] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f001f06ac5e
[ 989.951208] RDX: 0000000000000001 RSI: 000055d7e1e92760 RDI: 0000000000000007
[ 989.960015] RBP: 00007ffe03120970 R08: 0000000000000000 R09: 0000000000000000
[ 989.968825] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f001ef05e80
[ 989.977635] R13: 000055d7e1e92760 R14: 0000000000000007 R15: 00007f001f76bb98
[ 989.986451]
[ 989.989725] INFO: task python3:5440 is blocked on a mutex likely owned by task python3:3516.
[ 989.999999] task:python3 state:D stack:25312 pid:3516 tgid:3516 ppid:3515 task_flags:0x400100 flags:0x00080802
[ 990.013200] Call Trace:
[ 990.016740]
[ 990.019889] __schedule+0xad9/0x1490
[ 990.024703] ? io_schedule_timeout+0x130/0x130
[ 990.030489] ? find_held_lock+0x2b/0x80
[ 990.035594] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.041774] schedule+0xd4/0x260
[ 990.046194] schedule_timeout+0x103/0x260
[ 990.051492] ? rcu_is_watching+0x15/0xd0
[ 990.056690] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.063061] ? hrtimers_cpu_dying+0x250/0x250
[ 990.068751] ? ww_mutex_lock+0x160/0x160
[ 990.073951] msleep+0x1b/0x30
[ 990.078068] napi_set_threaded+0x376/0x410
[ 990.083464] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.088958] netif_set_threaded+0x254/0x400
[ 990.094453] ? sysfs_file_ops+0x140/0x140
[ 990.099749] threaded_store+0x1bb/0x220
[ 990.104853] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.110344] ? sysfs_file_ops+0x140/0x140
[ 990.115642] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.120928] ? sysfs_kf_write+0x65/0x160
[ 990.126127] kernfs_fop_write_iter+0x335/0x510
[ 990.131913] ? kernfs_ops+0xf0/0xf0
[ 990.136614] new_sync_write+0x308/0x6f0
[ 990.141717] ? new_sync_read+0x6f0/0x6f0
[ 990.146916] ? lock_acquire.part.0+0xbc/0x260
[ 990.152604] ? ksys_write+0xff/0x200
[ 990.157416] vfs_write+0x62a/0x980
[ 990.162031] ksys_write+0xff/0x200
[ 990.166637] ? __ia32_sys_read+0xc0/0xc0
[ 990.171834] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.177424] ? rcu_is_watching+0x15/0xd0
[ 990.182623] do_syscall_64+0xf3/0x5e0
[ 990.187520] ? exc_page_fault+0x9e/0xb0
[ 990.192624] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.199090] RIP: 0033:0x7feb2c76fc5e
[ 990.203900] RSP: 002b:00007ffdb4617570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.213200] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007feb2c76fc5e
[ 990.222007] RDX: 0000000000000001 RSI: 000055d54b9d4760 RDI: 0000000000000007
[ 990.230817] RBP: 00007ffdb4617580 R08: 0000000000000000 R09: 0000000000000000
[ 990.239627] R10: 0000000000000000 R11: 0000000000000202 R12: 00007feb2c60ae80
[ 990.248436] R13: 000055d54b9d4760 R14: 0000000000000007 R15: 00007feb2c59a480
[ 990.257253]
[ 990.260511] \x0aShowing all locks held in the system:
[ 990.269049] 1 lock held by khungtaskd/185:
[ 990.274442] #0: ffffffff99ece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.285915] 3 locks held by systemd-journal/803:
[ 990.291896] #0: ff1100180e7b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 990.301783] #1: ffffffff9a0c9340 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.314592] #2: ff1100019eb12c80 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 990.326054] 4 locks held by python3/3516:
[ 990.331349] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.341142] #1: ff1100015c343288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.352104] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.363251] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.373347] 4 locks held by python3/5440:
[ 990.378645] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.388438] #1: ff110001dcbd5688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.399397] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.410555] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.420642]
[ 990.423108] =============================================\x0a
[ 1014.290471] kworker/u130:3 (533) used greatest stack depth: 21192 bytes left
[ 1112.596483] INFO: task python3:5440 blocked for more than 245 seconds.
[ 1112.604633] Not tainted 7.0.0-rc6-qbvg-ga64f8f00fd9b #1
[ 1112.611883] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.621475] task:python3 state:D stack:25312 pid:5440 tgid:5440 ppid:5439 task_flags:0x400100 flags:0x00080000
[ 1112.634679] Call Trace:
[ 1112.638218]
[ 1112.641369] __schedule+0xad9/0x1490
[ 1112.646186] ? io_schedule_timeout+0x130/0x130
[ 1112.651974] ? find_held_lock+0x2b/0x80
[ 1112.657082] ? __lock_release.isra.0+0x59/0x170
[ 1112.662965] schedule+0xd4/0x260
[ 1112.667383] schedule_preempt_disabled+0x18/0x30
[ 1112.673365] __mutex_lock+0xd76/0x1f20
[ 1112.678373] ? threaded_store+0x132/0x220
[ 1112.683672] ? ww_mutex_lock+0x160/0x160
[ 1112.688871] ? selinux_inode_free_security+0x310/0x310
[ 1112.695442] ? cap_capable+0x1ed/0x3f0
[ 1112.700451] ? sysfs_file_ops+0x140/0x140
[ 1112.705753] ? threaded_store+0x132/0x220
[ 1112.711051] threaded_store+0x132/0x220
[ 1112.716147] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.721643] ? sysfs_file_ops+0x140/0x140
[ 1112.726939] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.732236] ? sysfs_kf_write+0x65/0x160
[ 1112.737439] kernfs_fop_write_iter+0x335/0x510
[ 1112.743223] ? kernfs_ops+0xf0/0xf0
[ 1112.747934] new_sync_write+0x308/0x6f0
[ 1112.753037] ? new_sync_read+0x6f0/0x6f0
[ 1112.758240] ? lock_acquire.part.0+0xbc/0x260
[ 1112.763927] ? ksys_write+0xff/0x200
[ 1112.768729] vfs_write+0x62a/0x980
[ 1112.773345] ksys_write+0xff/0x200
[ 1112.777952] ? __ia32_sys_read+0xc0/0xc0
[ 1112.783152] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.788735] ? rcu_is_watching+0x15/0xd0
[ 1112.793939] do_syscall_64+0xf3/0x5e0
[ 1112.798848] ? exc_page_fault+0x9e/0xb0
[ 1112.803949] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.810409] RIP: 0033:0x7f001f06ac5e
[ 1112.815219] RSP: 002b:00007ffe03120960 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.824509] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f001f06ac5e
[ 1112.833320] RDX: 0000000000000001 RSI: 000055d7e1e92760 RDI: 0000000000000007
[ 1112.842129] RBP: 00007ffe03120970 R08: 0000000000000000 R09: 0000000000000000
[ 1112.850935] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f001ef05e80
[ 1112.859745] R13: 000055d7e1e92760 R14: 0000000000000007 R15: 00007f001f76bb98
[ 1112.868560]
[ 1112.871867] INFO: task python3:5440 is blocked on a mutex likely owned by task python3:3516.
[ 1112.882141] task:python3 state:D stack:25312 pid:3516 tgid:3516 ppid:3515 task_flags:0x400100 flags:0x00080802
[ 1112.895342] Call Trace:
[ 1112.898880]
[ 1112.902032] __schedule+0xad9/0x1490
[ 1112.906843] ? io_schedule_timeout+0x130/0x130
[ 1112.912626] ? find_held_lock+0x2b/0x80
[ 1112.917801] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.923983] schedule+0xd4/0x260
[ 1112.928402] schedule_timeout+0x103/0x260
[ 1112.933691] ? rcu_is_watching+0x15/0xd0
[ 1112.938889] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.945262] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.950953] ? ww_mutex_lock+0x160/0x160
[ 1112.956154] msleep+0x1b/0x30
[ 1112.960351] napi_set_threaded+0x376/0x410
[ 1112.965745] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.971239] netif_set_threaded+0x254/0x400
[ 1112.976733] ? sysfs_file_ops+0x140/0x140
[ 1112.982032] threaded_store+0x1bb/0x220
[ 1112.987135] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.992627] ? sysfs_file_ops+0x140/0x140
[ 1112.997923] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.003222] ? sysfs_kf_write+0x65/0x160
[ 1113.008423] kernfs_fop_write_iter+0x335/0x510
[ 1113.014209] ? kernfs_ops+0xf0/0xf0
[ 1113.018923] new_sync_write+0x308/0x6f0
[ 1113.024024] ? new_sync_read+0x6f0/0x6f0
[ 1113.029225] ? lock_acquire.part.0+0xbc/0x260
[ 1113.034911] ? ksys_write+0xff/0x200
[ 1113.039723] vfs_write+0x62a/0x980
[ 1113.044349] ksys_write+0xff/0x200
[ 1113.048967] ? __ia32_sys_read+0xc0/0xc0
[ 1113.054167] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.059759] ? rcu_is_watching+0x15/0xd0
[ 1113.064958] do_syscall_64+0xf3/0x5e0
[ 1113.069856] ? exc_page_fault+0x9e/0xb0
[ 1113.074961] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.081427] RIP: 0033:0x7feb2c76fc5e
[ 1113.086237] RSP: 002b:00007ffdb4617570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.095534] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007feb2c76fc5e
[ 1113.104345] RDX: 0000000000000001 RSI: 000055d54b9d4760 RDI: 0000000000000007
[ 1113.113152] RBP: 00007ffdb4617580 R08: 0000000000000000 R09: 0000000000000000
[ 1113.121960] R10: 0000000000000000 R11: 0000000000000202 R12: 00007feb2c60ae80
[ 1113.130759] R13: 000055d54b9d4760 R14: 0000000000000007 R15: 00007feb2c59a480
[ 1113.139565]
[ 1113.142833] \x0aShowing all locks held in the system:
[ 1113.151372] 1 lock held by khungtaskd/185:
[ 1113.156768] #0: ffffffff99ece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.168245] 4 locks held by python3/3516:
[ 1113.173540] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.183331] #1: ff1100015c343288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.194298] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.205457] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.215551] 4 locks held by python3/5440:
[ 1113.220846] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.230637] #1: ff110001dcbd5688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.241599] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.252751] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.262847]
[ 1113.265317] =============================================\x0a
[ 1235.476782] INFO: task python3:5440 blocked for more than 368 seconds.
[ 1235.484930] Not tainted 7.0.0-rc6-qbvg-ga64f8f00fd9b #1
[ 1235.492179] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.501768] task:python3 state:D stack:25312 pid:5440 tgid:5440 ppid:5439 task_flags:0x400100 flags:0x00080002
[ 1235.514975] Call Trace:
[ 1235.518515]
[ 1235.521664] __schedule+0xad9/0x1490
[ 1235.526480] ? io_schedule_timeout+0x130/0x130
[ 1235.532264] ? find_held_lock+0x2b/0x80
[ 1235.537370] ? __lock_release.isra.0+0x59/0x170
[ 1235.543253] schedule+0xd4/0x260
[ 1235.547670] schedule_preempt_disabled+0x18/0x30
[ 1235.553649] __mutex_lock+0xd76/0x1f20
[ 1235.558655] ? threaded_store+0x132/0x220
[ 1235.563954] ? ww_mutex_lock+0x160/0x160
[ 1235.569151] ? selinux_inode_free_security+0x310/0x310
[ 1235.575718] ? cap_capable+0x1ed/0x3f0
[ 1235.580715] ? sysfs_file_ops+0x140/0x140
[ 1235.586017] ? threaded_store+0x132/0x220
[ 1235.591315] threaded_store+0x132/0x220
[ 1235.596416] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.601906] ? sysfs_file_ops+0x140/0x140
[ 1235.607204] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.612489] ? sysfs_kf_write+0x65/0x160
[ 1235.617689] kernfs_fop_write_iter+0x335/0x510
[ 1235.623475] ? kernfs_ops+0xf0/0xf0
[ 1235.628178] new_sync_write+0x308/0x6f0
[ 1235.633281] ? new_sync_read+0x6f0/0x6f0
[ 1235.638481] ? lock_acquire.part.0+0xbc/0x260
[ 1235.644157] ? ksys_write+0xff/0x200
[ 1235.648969] vfs_write+0x62a/0x980
[ 1235.653583] ksys_write+0xff/0x200
[ 1235.658195] ? __ia32_sys_read+0xc0/0xc0
[ 1235.663394] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.668977] ? rcu_is_watching+0x15/0xd0
[ 1235.674178] do_syscall_64+0xf3/0x5e0
[ 1235.679082] ? exc_page_fault+0x9e/0xb0
[ 1235.684184] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.690640] RIP: 0033:0x7f001f06ac5e
[ 1235.695446] RSP: 002b:00007ffe03120960 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.704745] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f001f06ac5e
[ 1235.713551] RDX: 0000000000000001 RSI: 000055d7e1e92760 RDI: 0000000000000007
[ 1235.722356] RBP: 00007ffe03120970 R08: 0000000000000000 R09: 0000000000000000
[ 1235.731160] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f001ef05e80
[ 1235.739966] R13: 000055d7e1e92760 R14: 0000000000000007 R15: 00007f001f76bb98
[ 1235.748777]
[ 1235.752041] INFO: task python3:5440 is blocked on a mutex likely owned by task python3:3516.
[ 1235.762312] task:python3 state:D stack:25312 pid:3516 tgid:3516 ppid:3515 task_flags:0x400100 flags:0x00080802
[ 1235.775514] Call Trace:
[ 1235.779050]
[ 1235.782199] __schedule+0xad9/0x1490
[ 1235.787006] ? io_schedule_timeout+0x130/0x130
[ 1235.792791] ? find_held_lock+0x2b/0x80
[ 1235.797883] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.804065] schedule+0xd4/0x260
[ 1235.808481] schedule_timeout+0x103/0x260
[ 1235.813778] ? rcu_is_watching+0x15/0xd0
[ 1235.818976] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.825344] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.831032] ? ww_mutex_lock+0x160/0x160
[ 1235.836223] msleep+0x1b/0x30
[ 1235.840348] napi_set_threaded+0x376/0x410
[ 1235.845742] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.851235] netif_set_threaded+0x254/0x400
[ 1235.856715] ? sysfs_file_ops+0x140/0x140
[ 1235.862008] threaded_store+0x1bb/0x220
[ 1235.867109] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.872601] ? sysfs_file_ops+0x140/0x140
[ 1235.877888] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.883184] ? sysfs_kf_write+0x65/0x160
[ 1235.888385] kernfs_fop_write_iter+0x335/0x510
[ 1235.894158] ? kernfs_ops+0xf0/0xf0
[ 1235.898869] new_sync_write+0x308/0x6f0
[ 1235.903973] ? new_sync_read+0x6f0/0x6f0
[ 1235.909170] ? lock_acquire.part.0+0xbc/0x260
[ 1235.914854] ? ksys_write+0xff/0x200
[ 1235.919664] vfs_write+0x62a/0x980
[ 1235.924267] ksys_write+0xff/0x200
[ 1235.928878] ? __ia32_sys_read+0xc0/0xc0
[ 1235.934077] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.939667] ? rcu_is_watching+0x15/0xd0
[ 1235.944868] do_syscall_64+0xf3/0x5e0
[ 1235.949776] ? exc_page_fault+0x9e/0xb0
[ 1235.954877] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.961343] RIP: 0033:0x7feb2c76fc5e
[ 1235.966150] RSP: 002b:00007ffdb4617570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.975447] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007feb2c76fc5e
[ 1235.984253] RDX: 0000000000000001 RSI: 000055d54b9d4760 RDI: 0000000000000007
[ 1235.993060] RBP: 00007ffdb4617580 R08: 0000000000000000 R09: 0000000000000000
[ 1236.001867] R10: 0000000000000000 R11: 0000000000000202 R12: 00007feb2c60ae80
[ 1236.010673] R13: 000055d54b9d4760 R14: 0000000000000007 R15: 00007feb2c59a480
[ 1236.019484]
[ 1236.022751] \x0aShowing all locks held in the system:
[ 1236.031288] 1 lock held by khungtaskd/185:
[ 1236.036681] #0: ffffffff99ece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.048151] 3 locks held by systemd-journal/803:
[ 1236.054126] #0: ff1100180eb372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 1236.064013] #1: ffffffff9a0c9340 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.076829] #2: ff110001697ff360 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0xf4/0x280
[ 1236.088290] 4 locks held by python3/3516:
[ 1236.093585] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.103372] #1: ff1100015c343288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.114335] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.125490] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.135583] 4 locks held by python3/5440:
[ 1236.140879] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.150669] #1: ff110001dcbd5688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.161626] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.172782] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.182872]
[ 1236.185336] =============================================\x0a
[ 1358.359076] INFO: task python3:5440 blocked for more than 491 seconds.
[ 1358.367225] Not tainted 7.0.0-rc6-qbvg-ga64f8f00fd9b #1
[ 1358.374474] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.384065] task:python3 state:D stack:25312 pid:5440 tgid:5440 ppid:5439 task_flags:0x400100 flags:0x00080002
[ 1358.397271] Call Trace:
[ 1358.400808]
[ 1358.403956] __schedule+0xad9/0x1490
[ 1358.408772] ? io_schedule_timeout+0x130/0x130
[ 1358.414558] ? find_held_lock+0x2b/0x80
[ 1358.419662] ? __lock_release.isra.0+0x59/0x170
[ 1358.425547] schedule+0xd4/0x260
[ 1358.429966] schedule_preempt_disabled+0x18/0x30
[ 1358.435945] __mutex_lock+0xd76/0x1f20
[ 1358.440949] ? threaded_store+0x132/0x220
[ 1358.446251] ? ww_mutex_lock+0x160/0x160
[ 1358.451449] ? selinux_inode_free_security+0x310/0x310
[ 1358.458018] ? cap_capable+0x1ed/0x3f0
[ 1358.463025] ? sysfs_file_ops+0x140/0x140
[ 1358.468324] ? threaded_store+0x132/0x220
[ 1358.473621] threaded_store+0x132/0x220
[ 1358.478725] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.484220] ? sysfs_file_ops+0x140/0x140
[ 1358.489516] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.494810] ? sysfs_kf_write+0x65/0x160
[ 1358.500009] kernfs_fop_write_iter+0x335/0x510
[ 1358.505793] ? kernfs_ops+0xf0/0xf0
[ 1358.510504] new_sync_write+0x308/0x6f0
[ 1358.515609] ? new_sync_read+0x6f0/0x6f0
[ 1358.520808] ? lock_acquire.part.0+0xbc/0x260
[ 1358.526493] ? ksys_write+0xff/0x200
[ 1358.531303] vfs_write+0x62a/0x980
[ 1358.535922] ksys_write+0xff/0x200
[ 1358.540536] ? __ia32_sys_read+0xc0/0xc0
[ 1358.545734] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.551323] ? rcu_is_watching+0x15/0xd0
[ 1358.556523] do_syscall_64+0xf3/0x5e0
[ 1358.561419] ? exc_page_fault+0x9e/0xb0
[ 1358.566518] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.572984] RIP: 0033:0x7f001f06ac5e
[ 1358.577793] RSP: 002b:00007ffe03120960 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.587088] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f001f06ac5e
[ 1358.595897] RDX: 0000000000000001 RSI: 000055d7e1e92760 RDI: 0000000000000007
[ 1358.604703] RBP: 00007ffe03120970 R08: 0000000000000000 R09: 0000000000000000
[ 1358.613507] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f001ef05e80
[ 1358.622317] R13: 000055d7e1e92760 R14: 0000000000000007 R15: 00007f001f76bb98
[ 1358.631128]
[ 1358.634405] INFO: task python3:5440 is blocked on a mutex likely owned by task python3:3516.
[ 1358.644675] task:python3 state:D stack:25312 pid:3516 tgid:3516 ppid:3515 task_flags:0x400100 flags:0x00080802
[ 1358.657877] Call Trace:
[ 1358.661415]
[ 1358.664564] __schedule+0xad9/0x1490
[ 1358.669376] ? io_schedule_timeout+0x130/0x130
[ 1358.675160] ? find_held_lock+0x2b/0x80
[ 1358.680255] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.686434] schedule+0xd4/0x260
[ 1358.690852] schedule_timeout+0x103/0x260
[ 1358.696148] ? rcu_is_watching+0x15/0xd0
[ 1358.701345] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.707715] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.713407] ? ww_mutex_lock+0x160/0x160
[ 1358.718606] msleep+0x1b/0x30
[ 1358.722732] napi_set_threaded+0x376/0x410
[ 1358.728128] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.733620] netif_set_threaded+0x254/0x400
[ 1358.739113] ? sysfs_file_ops+0x140/0x140
[ 1358.744407] threaded_store+0x1bb/0x220
[ 1358.749509] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.754998] ? sysfs_file_ops+0x140/0x140
[ 1358.760291] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.765587] ? sysfs_kf_write+0x65/0x160
[ 1358.770787] kernfs_fop_write_iter+0x335/0x510
[ 1358.776562] ? kernfs_ops+0xf0/0xf0
[ 1358.781271] new_sync_write+0x308/0x6f0
[ 1358.786371] ? new_sync_read+0x6f0/0x6f0
[ 1358.791573] ? lock_acquire.part.0+0xbc/0x260
[ 1358.797260] ? ksys_write+0xff/0x200
[ 1358.802070] vfs_write+0x62a/0x980
[ 1358.806677] ksys_write+0xff/0x200
[ 1358.811288] ? __ia32_sys_read+0xc0/0xc0
[ 1358.816486] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.822072] ? rcu_is_watching+0x15/0xd0
[ 1358.827271] do_syscall_64+0xf3/0x5e0
[ 1358.832166] ? exc_page_fault+0x9e/0xb0
[ 1358.837265] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.843730] RIP: 0033:0x7feb2c76fc5e
[ 1358.848535] RSP: 002b:00007ffdb4617570 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.857829] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007feb2c76fc5e
[ 1358.866634] RDX: 0000000000000001 RSI: 000055d54b9d4760 RDI: 0000000000000007
[ 1358.875440] RBP: 00007ffdb4617580 R08: 0000000000000000 R09: 0000000000000000
[ 1358.884247] R10: 0000000000000000 R11: 0000000000000202 R12: 00007feb2c60ae80
[ 1358.893043] R13: 000055d54b9d4760 R14: 0000000000000007 R15: 00007feb2c59a480
[ 1358.901856]
[ 1358.905123] \x0aShowing all locks held in the system:
[ 1358.913657] 1 lock held by khungtaskd/185:
[ 1358.919049] #0: ffffffff99ece120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.930529] 4 locks held by python3/3516:
[ 1358.935821] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.945612] #1: ff1100015c343288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.956573] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.967730] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.977821] 4 locks held by python3/5440:
[ 1358.983115] #0: ff1100012d2613f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.992905] #1: ff110001dcbd5688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.003857] #2: ff1100011fcf30f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.015014] #3: ff1100011fcd1d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.025105]
[ 1359.027561] =============================================\x0a