Sync jobs now use the same launch and reconcile architecture as backup jobs:
Launch rclone operations on Fly.io and return immediately.
Persist operation IDs in the database.
Let reconcile-stale-jobs detect terminal completion/failure asynchronously.
This avoids blocking API requests on long-running sync operations (including bisync) and prevents serverless timeout failures.
Why This Pattern
Vercel serverless functions have strict execution limits, while sync operations can run for many minutes. Waiting for terminal completion inside request handlers is not reliable for production workloads.
Current Flow
One-Time Sync (schedule = "none")
Route: src/app/api/jobs/sync/route.ts.
Creates the job and returns 201 immediately.
Uses after() to launch executeSyncJob() in background.
Uses a 300-second function budget for the background launch window, matching.
cron and transfer routes so broad selected-item syncs can record operation
IDs for every selected source before Vercel stops the function.
Does not recursively enumerate source or destination folders during the create request; quota and file-size checks run in the launch phase before Fly.io rclone starts.
UI behavior: the sync dialog closes after queueing, and users monitor progress/failures from Jobs and Activity Logs.
Uses after() to launch executeSyncJob() for each due job.
Returns cron response immediately after queueing launch work.
Uses the same 300-second launch budget as run-now sync and transfer routes so.
broad selected-item syncs can record every operation ID before reconciliation
checks completeness.
Launch Phase
Module: src/lib/sync/execute-sync-job.ts.
Validates job state and payload.
Resolves/validates source and destination paths.
Uses OneDrive provider metadata as a fast-launch signal for large one-way.
sync folders. When the estimate is reliable and above the deferred-manifest
threshold, recursive manifest validation is skipped for the launch so the
Fly.io rclone operation ID is persisted before Vercel can time out.
Preserves two-way bisync logic.
resync flag for first-time bisync initialization.
--check-access is explicit opt-in only (checkAccess: true).
Reason: avoid false RCLONE_TEST aborts on valid runs (for example, after deletions that leave both sides empty).
Launches operations via executeSyncOperation().
Stores launched operation IDs incrementally on the job record.
Finalizes job/item status and activity logs when operations reach terminal state.
Bisync Auto-Recovery Enhancements
Conflict detection + metadata: reconcile-stale-jobs.ts now uses isBisyncConflict() to identify bisync-specific failures. Failed jobs reset bisyncInitialized, update a recovery_attempts counter (default 3 attempts, configurable via SYNC_MAX_RECOVERY_ATTEMPTS), and store metadata.requiresResync = true/resyncReason so the next launch uses --resync.
Executor awareness: execute-sync-job.ts inspects metadata.requiresResync alongside bisyncInitialized to set needsResync, logs the resync reason (first-time initialization vs recovery from bisync conflict), and forwards metadata to the Fly.io sync service.
Fly.io rclone logging: fly-rclone/src/routes/syncRoutes.js now prefers --resync over --check-access, logs whether metadata.resyncReason drove the decision, and surfaces the flag configuration for every bisync operation.
Metadata cleanup: After a successful reconciliation run that used resync, the reconciler clears requiresResync so subsequent launches revert to normal two-way behavior (only --resync when bisyncInitialized=false again).
Large Sync Watchdog Profiles
Sync launch now classifies each operation as one of.
normal-sync.
large-one-way-sync.
large-two-way-sync.
large-bisync-initialization.
Classification uses estimated size/file/folder thresholds and unknown-root fallback. This protects very large provider folders where listing/checking can stay byte-idle for long periods.
First-time bisync initialization (--resync) is handled conservatively and receives the longest timeout profile.
Sync metadata now forwards watchdog hints (operationTimeoutMs, stallTimeoutMs, profile label/reason, sync mode, and recovery/init flags) to Fly rclone so queue defaults do not incorrectly cancel valid long-running operations.
Timeout cancellations remain cancelled at operation level with cancelReason = timeout; reconciliation treats those as failed timeout outcomes rather than user cancellations.
API Behavior
/api/jobs/sync
Returns success immediately after creating/queueing launch work. It does not block on sync completion.
The create request also avoids recursive provider manifest resolution. Large OneDrive sync launches can defer recursive manifest validation when provider metadata supplies a reliable size estimate, preventing No operation IDs recorded; job orphaned failures caused by pre-launch enumeration exceeding the serverless window.
/api/rclone/sync
Launches a sync operation and returns the operation ID immediately. It does not poll for terminal completion in the request lifecycle.
Google->Google Cross-Account Guard
File: fly-rclone/src/routes/syncRoutes.js.
One-way sync no longer forces --drive-server-side-across-configs.
The flag is now explicit opt-in only (options.enableDriveServerSideAcrossConfigs === true).
Reason: production logs showed repeated googleapi: Error 404: File not found failures for cross-account Google sync jobs when this flag was forced.