Skip to content

fix(federation): persist address book sync token after each page - #65941

Merged
SebastianKrupinski merged 1 commit into
masterfrom
fix/federation-sync-part-2-save-token-after-each-page
Nov 19, 2026
Merged

SebastianKrupinski merged 1 commit into
masterfrom
fix/federation-sync-part-2-save-token-after-each-page

Conversation

@SebastianKrupinski

@SebastianKrupinski SebastianKrupinski commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Resolves: # client ticket
  • When synchronizing large system address books with truncated results a failure on the 3rd page never saves the sync token, causing the sync to restart from the beginning, this saves the token after each page, so a failure continues where it stopped

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@SebastianKrupinski SebastianKrupinski added this to the Nextcloud 36 milestone Oct 26, 2026
@SebastianKrupinski SebastianKrupinski self-assigned this Oct 26, 2026
@SebastianKrupinski
SebastianKrupinski requested a review from a team as a code owner October 26, 2026 16:42
@SebastianKrupinski SebastianKrupinski added the 3. to review Waiting for reviews label Oct 26, 2026
@SebastianKrupinski
SebastianKrupinski requested review from Altahrim, leftybournes, provokateurin and sorbaugh and removed request for a team October 26, 2026 16:42
@github-project-automation github-project-automation Bot moved this to 🏗️ In progress in 💌 📅 👥 Groupware team Oct 26, 2026
@DerDreschner

Copy link
Copy Markdown
Member

No backports? 😄

@DerDreschner DerDreschner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a rebase: #65224 added IAppConfig $appConfig as the fourth constructor argument of SyncFederationAddressBooks, so the two new tests fail with a TypeError on current master, even though the merge has no conflicts (inline).

Outside the diff:

  • SyncFederationAddressBooks.php:79: if the final token equals $oldSyncToken, nothing is written, so the last intermediate token from setServerSyncToken() stays stored while the status says OK. This happens on a --full sync of a truncated address book when the remote hasn't changed: stored …/T, the pages save …/init_2500_T and …/init_5000_T, the last page returns …/T, and …/init_5000_T stays. The next SyncJob then downloads every card after id 5000 again. Comparing against the last saved token (e.g. $oldSyncToken = $syncToken after saving) would fix it.
  • SyncFederationAddressBooks.php:90: on a 401, SyncService::syncRemoteAddressBook() deletes the local address book, but the stored intermediate token stays. Before, a first sync that hit this kept a null token and started over. Now, once access is restored, the sync creates an empty book and resumes from init_<lastId>_…, so the contacts from the earlier pages are missing until a manual --full. The token should be cleared when the address book is deleted.

Assisted by ClaudeCode:claude-opus-5-5

->willReturnOnConsecutiveCalls(['init_1_5', true], ['init_2_5', true], ['5', false]);

/** @var SyncService $syncService */
$s = new SyncFederationAddressBooks($dbHandler, $syncService, $this->discoveryService, $this->logger);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a rebase this needs $this->createMock(IAppConfig::class) before $this->logger (#65224 added IAppConfig as the fourth constructor argument). Same at line 145.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

$targetPrincipal,
$targetBookProperties
);
if ($truncated && $syncToken !== null) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With --full, saving intermediate tokens can lose remote deletions. If a full sync fails after the first page, init_2500_T is stored. The next regular SyncJob resumes the init listing, which only reports cards that still exist, and deletePendingCards() only runs with $full. Before this change, the next run synced from the old token and got the deletions from the change log.

Example: stored token T_old, the remote deletes carol's card, an admin runs occ federation:sync-addressbooks --full on a 6000-card book, page 1 saves init_2500_T, page 2 fails. The next SyncJob ends at T without seeing carol's deletion, so her card stays until another full sync succeeds.

Not saving intermediate tokens when $full is set would avoid this; a --full retry starts from null anyway.

Tests: nothing covers a truncated page with a null token or a $full sync with truncated pages, so removing the null check or saving tokens during --full keeps the tests green.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

// Keep the progress of completed pages in case a later page fails
$this->dbHandler->setServerSyncToken($url, $syncToken);
}
} while ($truncated);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-existing, but with this PR it also writes to the DB on every pass: do … while ($truncated) never ends if the remote keeps answering truncated without moving the token forward. The remote's incremental query uses gte('synctoken', …), so with carddav_sync_request_truncation set to 1 it returns the token it was given. A null token with truncated = true loops as well. Failing the sync when the new token is null or equal to the previous one would stop it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is out of scope but I fixed it because it can be a big issue.

$this->assertSame(TrustedServers::STATUS_OK, (int)$result[0]['status']);
}

public function testSetServerSyncToken(): void {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With only one server in the table, this would still pass if setServerSyncToken() updated every row. A second server whose sync_token stays null would cover the url_hash condition.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@SebastianKrupinski
SebastianKrupinski force-pushed the fix/federation-sync-part-2-save-token-after-each-page branch from d83934b to 84a8c77 Compare November 19, 2026 19:25
@SebastianKrupinski

Copy link
Copy Markdown
Contributor Author

/backport to stable35

@SebastianKrupinski

Copy link
Copy Markdown
Contributor Author

/backport to stable34

@SebastianKrupinski

Copy link
Copy Markdown
Contributor Author

/backport to stable33

Signed-off-by: SebastianKrupinski <krupinskis05@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SebastianKrupinski
SebastianKrupinski force-pushed the fix/federation-sync-part-2-save-token-after-each-page branch from b7f46fc to a47eea8 Compare November 19, 2026 19:45
@SebastianKrupinski
SebastianKrupinski merged commit c5c9b14 into master Nov 19, 2026
155 checks passed
@SebastianKrupinski
SebastianKrupinski deleted the fix/federation-sync-part-2-save-token-after-each-page branch November 19, 2026 20:00
@github-project-automation github-project-automation Bot moved this from 🏗️ In progress to ☑️ Done in 💌 📅 👥 Groupware team Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews AI assisted

Projects

Status: ☑️ Done

Development

Successfully merging this pull request may close these issues.

4 participants