Repository navigation
fix(config): save config.toml atomically and serialise its updates - #562
vitorcamillo wants to merge 8 commits into
Conversation
pair(), unpair(), update_client_settings(), update_profiles() and the config migration rewrite config.toml in place (truncate + write). A crash, a full disk or a concurrent reader in between sees an empty or partially written file; after a crash Wolf then fails to load it on the next start and paired clients/profiles have to be recovered by hand. Write the new content to a temporary file in the same directory, fsync it, copy the original mode and owner, rename it over the destination and fsync the directory. Symlinks are followed so the link target is updated. When the file can't be replaced atomically (directory not writable, ownership we can't reproduce, or config.toml bind mounted as a single file: rename() fails with EBUSY) fall back to the previous in-place write with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pair(), unpair(), update_client_settings() and update_profiles() load the whole config.toml, change their part and save it back without any synchronisation, so two concurrent API calls start from the same file and the last writer silently drops the other change. Hold a mutex, shared by all the copies of the Config, for the duration of the file round-trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I think there's value in fixing the concurrent updates and potential config corruption, but I'm not convinced this implementation is the right approach. We're adding quite a bit of low-level filesystem handling for what should be a relatively simple operation, and the non-atomic fallback effectively reintroduces the original problem. We also still update in-memory state before persistence succeeds, so failures can leave the two inconsistent. I'd prefer we keep the synchronization fix and simplify the atomic write implementation around Wolf's actual deployment requirements. The underlying issues are worth fixing, but I don't think the additional complexity is justified as it stands. |
…mory Address review feedback on games-on-whales#562: - write config.toml to `<file>.tmp` and rename it over the original, with no non-atomic fallback; a failed save now throws instead of truncating - drop the permission/ownership/symlink handling and the fsync calls - pair/unpair/update_client_settings/update_profiles save the file first and only then update the in-memory state, so a failed save leaves both untouched (new test case) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, that makes sense. I've pushed a simpler version (2796376):
The source change against |
Resolve the conflict in tests/testMoonlight.cpp by keeping both new test cases: the atomic config save one from this branch and the unknown-issuer cert one from beta. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I merged |
|
Tested on an NVIDIA host (GA102):
On beta, two concurrent Still open: writes to the same section are lost, more often than on beta. The profile endpoints call - state::update_profiles(
- state_->app_state->config,
- profiles.push_back(rfl::Reflector<events::Profile>::to(p, this->state_->app_state->event_bus)));
+ state::update_profiles(state_->app_state->config, [&](const state::ProfilesList &profiles) {
+ return profiles.push_back(rfl::Reflector<events::Profile>::to(p, this->state_->app_state->event_bus));
+ });With it, every row above is 0 lost (n=100, pairing n=15). A new unit test (4 threads x 10 profile adds, plus run_uid and run_gid on one client) fails 12 == 42 when We have a test bench (one AMD host with a discrete and an integrated GPU, one NVIDIA host, scripted Moonlight clients) and are glad to take on A/B runs or side quests. GPU age is the one limit worth naming. |
…ted config.toml - update_profiles/update_client_settings take a function of the current state, applied while holding config_file_mutex, so concurrent writes to the same section are no longer lost; the API endpoints use it - fall back to an in-place write when config.toml is a bind-mount point (rename fails with EBUSY/EXDEV) - keep the rename error code when formatting the error message Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps the previous file content in memory and the complete new content in config.toml.tmp until the in-place write succeeds, so a failed write on a bind-mounted config.toml never leaves a truncated file without a copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Revisor-Worker: muse
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks a lot for the thorough test report and the bench, @andrewdunndev, it was really helpful. I've pushed fixes for the three points:
Tests added:
CI is green on my fork (build + tests on g++/clang, shared/static, x86_64/aarch64, and clang-format). |
Problem
Every runtime change to the configuration (
pair,unpair,update_client_settings,update_profiles, and thev≤6 → v7 migration) rewrites
config.tomlin place:rfl::toml::save()/std::ofstreamtruncate the file and thenwrite it. Two consequences:
SIGKILL, power loss or full disk between the truncate and the end of the writeleaves an empty or partial
config.toml; Wolf then fails to load it on the next start and paired clients /profiles have to be recovered by hand. Any concurrent reader (Wolf itself while handling another API call,
wolf-ui, a backup job) can also observe the partial file.
calls running concurrently (e.g.
clients/settingsandprofiles/..., or a pairing while the UI saves profiles)each start from the same old file and the last writer silently drops the other change.
How to reproduce
update_profiles()while another thread readsconfig.toml: the reader sees content different from whatis being written (in our run on
beta: 1748 bad reads out of 100 writes).update_client_settings()andupdate_profiles()from two threads: the final file keeps only one of thetwo changes (
app_state_folderreverted to"some/folder").Both are covered by the new tests.
Fix
save_file_atomically(): write the new content to<config>.tmpnext to the file andrename()it overconfig.toml. There is no non-atomic fallback: if the write or the rename fails it throws and the original file isleft as it was. This assumes the config lives in a directory (the documented
/etc/wolfvolume), not a singlebind-mounted file.
Configgets astd::shared_ptr<std::mutex> config_file_mutex(shared by every copy of theConfig), held by thefour read-modify-write functions.
immer::atoms, so a failed save leavesboth the file and memory unchanged.
Tests
New
LocalState config is saved atomicallytest case intests/testMoonlight.cpp(works on a copy ofconfig.test.tomlin a temp dir):update_profiles()while a thread keeps reading the file; everyread must equal the expected content. Fails on
beta(1748 bad reads).update_client_settings()andupdate_profiles()(on a copy of theConfig) from two threads, 30 times each; both changes must be in the file.Fails without the mutex.
update_profiles()throws, the fileand the in-memory profiles are unchanged.
Built in the devbase container (GCC, Release, C++20):
~[DOCKER]suiteAll tests passed (617 assertions in 38 test cases), also withMALLOC_PERTURB_=165;clang-format18.1.8 clean.🤖 Generated with Claude Code