[ 2232.467526][ T70] [ 2232.467629][ T70] ====================================================== [ 2232.467790][ T70] WARNING: possible circular locking dependency detected [ 2232.467941][ T70] 7.3.0-rc3-virtme #1 Not tainted [ 2232.468060][ T70] ------------------------------------------------------ [ 2232.468211][ T70] kworker/u16:1/70 is trying to acquire lock: [ 2232.468364][ T70] ff11000008c64e10 (&dev_instance_lock_key#2){+.+.}-{4:4}, at: unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.468648][ T70] [ 2232.468648][ T70] but task is already holding lock: [ 2232.468829][ T70] ff1100001a101e10 (&dev_instance_lock_key){+.+.}-{4:4}, at: unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.469090][ T70] [ 2232.469090][ T70] which lock already depends on the new lock. [ 2232.469090][ T70] [ 2232.469322][ T70] [ 2232.469322][ T70] the existing dependency chain (in reverse order) is: [ 2232.469525][ T70] [ 2232.469525][ T70] -> #1 (&dev_instance_lock_key){+.+.}-{4:4}: [ 2232.469702][ T70] __lock_acquire+0x4ff/0x900 [ 2232.469824][ T70] lock_acquire.part.0+0xd4/0x280 [ 2232.469939][ T70] __mutex_lock+0x1ae/0x1f10 [ 2232.470055][ T70] unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.470200][ T70] default_device_exit_batch+0x543/0x820 [ 2232.470344][ T70] ops_undo_list+0x2ce/0x8f0 [ 2232.470460][ T70] cleanup_net+0x431/0x940 [ 2232.470577][ T70] process_one_work+0xe3e/0x1560 [ 2232.470700][ T70] worker_thread+0x4f1/0xd60 [ 2232.470817][ T70] kthread+0x367/0x460 [ 2232.470932][ T70] ret_from_fork+0x474/0x6b0 [ 2232.471052][ T70] ret_from_fork_asm+0x11/0x20 [ 2232.471170][ T70] [ 2232.471170][ T70] -> #0 (&dev_instance_lock_key#2){+.+.}-{4:4}: [ 2232.471349][ T70] check_prev_add+0xd8/0xe90 [ 2232.471468][ T70] validate_chain+0x6b8/0xc20 [ 2232.471582][ T70] __lock_acquire+0x4ff/0x900 [ 2232.471701][ T70] lock_acquire.part.0+0xd4/0x280 [ 2232.471817][ T70] __mutex_lock+0x1ae/0x1f10 [ 2232.471933][ T70] unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.472079][ T70] default_device_exit_batch+0x543/0x820 [ 2232.472221][ T70] ops_undo_list+0x2ce/0x8f0 [ 2232.472335][ T70] cleanup_net+0x431/0x940 [ 2232.472450][ T70] process_one_work+0xe3e/0x1560 [ 2232.472565][ T70] worker_thread+0x4f1/0xd60 [ 2232.472682][ T70] kthread+0x367/0x460 [ 2232.472796][ T70] ret_from_fork+0x474/0x6b0 [ 2232.472912][ T70] ret_from_fork_asm+0x11/0x20 [ 2232.473026][ T70] [ 2232.473026][ T70] other info that might help us debug this: [ 2232.473026][ T70] [ 2232.473258][ T70] Possible unsafe locking scenario: [ 2232.473258][ T70] [ 2232.473431][ T70] CPU0 CPU1 [ 2232.473554][ T70] ---- ---- [ 2232.473678][ T70] lock(&dev_instance_lock_key); [ 2232.473797][ T70] lock(&dev_instance_lock_key#2); [ 2232.473973][ T70] lock(&dev_instance_lock_key); [ 2232.474147][ T70] lock(&dev_instance_lock_key#2); [ 2232.474265][ T70] [ 2232.474265][ T70] *** DEADLOCK *** [ 2232.474265][ T70] [ 2232.474437][ T70] locks held by kworker/u16:1/70: 6, last CPU#0: [ 2232.474585][ T70] #0: ff11000001a21d40 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0xd9a/0x1560 [ 2232.474824][ T70] #1: ffa00000004b7d10 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xdfc/0x1560 [ 2232.475026][ T70] #2: ffffffff88d02968 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbf/0x940 [ 2232.475229][ T70] #3: ffffffff88d1e420 (rtnl_mutex){+.+.}-{4:4}, at: default_device_exit_batch+0xa8/0x820 [ 2232.475465][ T70] #4: ff1100001f851e10 (&dev_instance_lock_key#2){+.+.}-{4:4}, at: unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.475728][ T70] #5: ff1100001a101e10 (&dev_instance_lock_key){+.+.}-{4:4}, at: unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.475989][ T70] [ 2232.475989][ T70] stack backtrace: [ 2232.476135][ T70] CPU: 0 UID: 0 PID: 70 Comm: kworker/u16:1 Not tainted 7.3.0-rc3-virtme #1 PREEMPT(full) [ 2232.476138][ T70] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 2232.476140][ T70] Workqueue: netns cleanup_net [ 2232.476143][ T70] Call Trace: [ 2232.476145][ T70] [ 2232.476146][ T70] dump_stack_lvl+0x6f/0xa0 [ 2232.476149][ T70] print_circular_bug.cold+0x38/0x46 [ 2232.476153][ T70] check_noncircular+0x171/0x1a0 [ 2232.476156][ T70] check_prev_add+0xd8/0xe90 [ 2232.476158][ T70] ? lock_acquire.part.0+0xd4/0x280 [ 2232.476160][ T70] validate_chain+0x6b8/0xc20 [ 2232.476162][ T70] __lock_acquire+0x4ff/0x900 [ 2232.476164][ T70] lock_acquire.part.0+0xd4/0x280 [ 2232.476166][ T70] ? unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.476168][ T70] ? rcu_is_watching+0x16/0xd0 [ 2232.476171][ T70] ? lock_acquire+0x13c/0x160 [ 2232.476173][ T70] __mutex_lock+0x1ae/0x1f10 [ 2232.476174][ T70] ? unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.476176][ T70] ? unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.476178][ T70] ? __lock_acquire+0x4ff/0x900 [ 2232.476180][ T70] ? ww_mutex_lock+0x160/0x160 [ 2232.476181][ T70] ? find_held_lock+0x2b/0x80 [ 2232.476183][ T70] ? rcu_is_watching+0x16/0xd0 [ 2232.476184][ T70] ? mark_held_locks+0x40/0x70 [ 2232.476186][ T70] ? kfree+0x22/0x590 [ 2232.476189][ T70] ? lockdep_hardirqs_on_prepare.part.0+0x9a/0x160 [ 2232.476191][ T70] ? _raw_spin_unlock_irqrestore+0x53/0x80 [ 2232.476193][ T70] ? unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.476195][ T70] ? rtnl_is_locked+0x15/0x20 [ 2232.476197][ T70] unregister_netdevice_many_notify+0x3ef/0x1b40 [ 2232.476198][ T70] ? ops_undo_list+0x2ce/0x8f0 [ 2232.476200][ T70] ? worker_thread+0x4f1/0xd60 [ 2232.476201][ T70] ? kthread+0x367/0x460 [ 2232.476203][ T70] ? ret_from_fork+0x474/0x6b0 [ 2232.476204][ T70] ? ret_from_fork_asm+0x11/0x20 [ 2232.476206][ T70] ? unregister_netdevice_queued+0x80/0x80 [ 2232.476208][ T70] ? unregister_netdevice_many+0x20/0x20 [ 2232.476210][ T70] default_device_exit_batch+0x543/0x820 [ 2232.476212][ T70] ? unregister_netdevice_queue+0x470/0x470 [ 2232.476214][ T70] ? perf_trace_sched_switch+0x7c0/0x7c0 [ 2232.476216][ T70] ? fou_exit_net+0x30/0x110 [ 2232.476219][ T70] ops_undo_list+0x2ce/0x8f0 [ 2232.476221][ T70] ? lockdep_hardirqs_on+0x91/0x130 [ 2232.476223][ T70] ? netns_install+0x2a0/0x2a0 [ 2232.476225][ T70] ? __lock_release.isra.0+0x69/0x1a0 [ 2232.476227][ T70] ? idr_destroy+0x121/0x250 [ 2232.476230][ T70] cleanup_net+0x431/0x940 [ 2232.476232][ T70] ? net_passive_dec+0x230/0x230 [ 2232.476234][ T70] ? lock_acquire+0x13c/0x160 [ 2232.476236][ T70] ? rcu_is_watching+0x16/0xd0 [ 2232.476238][ T70] process_one_work+0xe3e/0x1560 [ 2232.476240][ T70] ? __queue_delayed_work+0x470/0x470 [ 2232.476241][ T70] ? lock_acquire.part.0+0xd4/0x280 [ 2232.476244][ T70] worker_thread+0x4f1/0xd60 [ 2232.476247][ T70] ? rescuer_thread+0x1340/0x1340 [ 2232.476248][ T70] kthread+0x367/0x460 [ 2232.476250][ T70] ? trace_irq_enable.constprop.0+0x9b/0x160 [ 2232.476253][ T70] ? kthread_affine_preferred+0x4c0/0x4c0 [ 2232.476255][ T70] ret_from_fork+0x474/0x6b0 [ 2232.476257][ T70] ? arch_exit_to_user_mode_prepare.isra.0+0x120/0x120 [ 2232.476259][ T70] ? __switch_to+0x5a3/0xe00 [ 2232.476261][ T70] ? kthread_affine_preferred+0x4c0/0x4c0 [ 2232.476263][ T70] ret_from_fork_asm+0x11/0x20 [ 2232.476266][ T70]