ocfs2: synchronize heartbeat callbacks with o2net teardown

In the Linux kernel, the following vulnerability has been resolved: ocfs2: synchronize heartbeat callbacks with o2net teardown Patch series "ocfs2: harden heartbeat teardown races". This series fixes two OCFS2 heartbeat/o2net teardown races found by KASAN. This patch (of 2): Heartbeat callbacks stay registered while configfs local-node teardown enters o2net_stop_listening(). A node-down event can still run through o2net_disconnect_node() and o2net_set_nn_state() while teardown is destroying o2net_wq, so the later queue/flush operations can hit a dead workqueue. KASAN has caught this as a slab-use-after-free in __queue_work() with the call chain: KASAN slab-use-after-free in __queue_work+0x56/0xa90 Read of size 4 Call trace: dump_stack_lvl+0x66/0xa0 print_report+0xce/0x630 __queue_work+0x56/0xa90 srso_alias_return_thunk+0x5/0xfbef5 __virt_addr_valid+0x19f/0x330 kasan_report+0xe0/0x110 __queue_delayed_work+0x58/0x1e0 queue_delayed_work_on+0xb4/0xc0 o2net_set_nn_state+0x467/0x840 o2net_disconnect_node+0x7b/0xe0 o2net_hb_node_down_cb+0x54/0x60 o2hb_run_event_list+0x236/0x2d0 o2hb_check_slot+0xad4/0xbc0 lock_release+0xc8/0x290 o2hb_check_slot+0x9ea/0xbc0 trace_hardirqs_on+0x18/0x130 o2hb_do_disk_heartbeat+0x646/0xb30 (fs/ocfs2/cluster/heartbeat.c:1079) __lock_acquire+0x466/0x2260 lockdep_hardirqs_on_prepare+0xea/0x1a0 ktime_get_with_offset+0xe9/0x230 o2hb_thread+0x14e/0x770 kthread+0x1ad/0x1f0 ret_from_fork+0x3c9/0x540 __switch_to+0x2e9/0x730 ret_from_fork_asm+0x1a/0x30 Allocated by task stack: kasan_save_stack+0x33/0x60 kasan_save_track+0x14/0x30 __kasan_kmalloc+0xaa/0xb0 __kmalloc_noprof+0x292/0x760 __alloc_workqueue+0x736/0xc60 alloc_workqueue_noprof+0xb1/0x110 o2net_start_listening+0xe5/0x430 o2nm_node_local_store+0x184/0x310 configfs_write_iter+0x18a/0x210 vfs_write+0x469/0x810 ksys_write+0xd2/0x170 do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87) entry_SYSCALL_64_after_hwframe+0x77/0x7f Freed by task stack: kasan_save_stack+0x33/0x60 kasan_save_track+0x14/0x30 kasan_save_free_info+0x3b/0x60 __kasan_slab_free+0x5f/0x80 kfree+0x313/0x590 rcu_core+0x4f4/0x1320 handle_softirqs+0x156/0x660 queue_delayed_work_on o2net_set_nn_state o2net_disconnect_node o2net_hb_node_down_cb o2hb_run_event_list Keep heartbeat callbacks registered so quorum state still tracks node state, but stop them from driving o2net reconnect/disconnect work once local teardown starts. Mark the transport offline before destroying o2net_wq, wait for any in-flight heartbeat callback to finish, and delay bring-up replay until the new local node is published through o2nm_this_node(). The replay also has to stay serialized with heartbeat callback delivery. Otherwise a live-node snapshot can be copied, a real hb_down callback can install -ENOTCONN for a peer, and the stale replay can call o2net_hb_node_up() for that same peer and queue reconnect work even though heartbeat is already down. The buggy scenario involves two paths, with each column showing the order within that path: local-node teardown: heartbeat node-down callback: 1. configfs local-off enters 1. o2hb_run_event_list() invokes o2net_stop_listening(). o2net_hb_node_down_cb(). 2. teardown heads for 2. the callback reaches destroy_workqueue(o2net_wq). o2net_disconnect_node() and o2net_set_nn_state(). 3. teardown destroys and NULLs 3. the callback flushes or queues o2net_wq. work through o2net_wq.

Quoted source text, attributed separately from HOL analysis.