# Unified Rclone Module

This module provides a comprehensive, unified interface for all rclone-related functionality including file transfers, file ID resolution, configuration management, and React hooks.

The unified rclone transfer service provides a single interface for all file and folder transfers using the existing rclone + Fly.io infrastructure. It includes proper TypeScript interfaces, error handling, and progress tracking callbacks.

## Features

- **Single File Transfer**: Transfer individual files between cloud services.
- **Folder Transfer**: Transfer entire folders with structure preservation.
- **Batch Transfer**: Transfer multiple files in a single batch operation.
- **Progress Tracking**: Real-time progress updates with callbacks.
- **Error Handling**: Comprehensive error handling with typed error codes.
- **Cancellation**: Cancel individual operations or all active transfers.
- **Status Monitoring**: Get real-time status of transfer operations.
- **Automatic File Filtering**: Skips files that cannot be properly transferred by rclone.

## File Filtering

The transfer service automatically filters out files that rclone cannot properly transfer between cloud storage services. This includes:

- **Symbolic Links & Shortcuts**: Windows .lnk files, macOS .alias files, .url files.
- **Platform-Specific Files**: iCloud .icloud placeholders, .DS_Store, Thumbs.db, resource forks.
- **Google Workspace Non-Exportable**: Google Forms, My Maps, Sites, Drive shortcuts.
- **System & Temporary Files**: Hidden files, .tmp files, backup files ending with ~.

Files are filtered based on:
- File extensions (e.g., `.lnk`, `.icloud`).
- MIME types (e.g., `application/vnd.google-apps.form`).
- Exact filenames (e.g., `.DS_Store`).
- Pattern matching (e.g., files starting with `._`).

For batch transfers, filtered files are skipped and reported in the result. For single file transfers, an error is thrown if the file cannot be transferred.

See `src/config/transfer-filters.ts` for the complete list of filter rules.

## Usage

### Basic File Transfer

```typescript
import { transferFile } from "~/lib/rclone";

const result = await transferFile({
  sourceService: "google",
  sourceAccountId: "default",
  sourceFileId: "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms",
  destService: "onedrive",
  destAccountId: "default",
  destFolderId: "root",
  fileName: "copied-file.xlsx",
  mimeType: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", // Optional: for better filtering
  fileSize: 1024000 // Optional: file size in bytes
}, {
  onStart: (metadata) => console.log("Transfer started:", metadata.operationId),
  onProgress: (metadata) => console.log("Progress:", metadata.progress?.percentage + "%"),
  onComplete: (metadata) => console.log("Transfer completed:", metadata.operationId),
  onError: (metadata, error) => console.error("Transfer failed:", error),
});
```

### Folder Transfer

```typescript
import { transferFolder } from "~/lib/rclone";

const result = await transferFolder({
  sourceService: "google",
  sourceAccountId: "default",
  sourceFolderId: "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms",
  destService: "dropbox",
  destAccountId: "default",
  destFolderId: "root",
  folderName: "My Documents",
  preserveStructure: true,
  dryRun: false
}, {
  onProgress: (metadata) => {
    const progress = metadata.progress;
    if (progress) {
      console.log(`Progress: ${progress.percentage}% (${progress.filesTransferred}/${progress.totalFiles} files)`);
    }
  },
  onComplete: (metadata) => console.log("Folder transfer completed"),
});
```

### Batch Transfer

```typescript
import { batchTransferFiles } from "~/lib/transfer/rclone-transfer-service";

const result = await batchTransferFiles({
  operations: [
    {
      sourceService: "google",
      sourceAccountId: "default",
      sourceFileId: "file1",
      destService: "onedrive",
      destAccountId: "default",
      destFolderId: "root",
    },
    {
      sourceService: "google",
      sourceAccountId: "default",
      sourceFileId: "file2",
      destService: "onedrive",
      destAccountId: "default",
      destFolderId: "root",
    },
  ]
}, {
  onStart: (metadata) => console.log(`Operation ${metadata.data?.batchIndex} started`),
  onProgress: (metadata) => console.log(`Operation ${metadata.operationId} progress: ${metadata.progress?.percentage}%`),
  onComplete: (metadata) => console.log(`Operation ${metadata.operationId} completed`),
});

// Check for skipped files in batch transfers
if (result.skippedFiles && result.skippedFiles.length > 0) {
  console.log(`${result.skippedFiles.length} files were skipped:`);
  result.skippedFiles.forEach(file => {
    console.log(`- ${file.fileName}: ${file.skipReason}`);
  });
}
```

### Using the Service Class Directly

```typescript
import { RcloneTransferService } from "~/lib/rclone";

// Create a custom service instance with different configuration
const transferService = new RcloneTransferService({
  defaultPollInterval: 1000, // Poll every second
  maxRetries: 5,
  retryDelay: 2000,
  timeout: 60000,
});

// Use the service
const result = await transferService.transferFile(request, callbacks);

// Get all active transfers
const activeTransfers = transferService.getActiveTransfers();

// Cancel all transfers
await transferService.cancelAllTransfers();

// Clean up when done
transferService.dispose();
```

### Monitoring Transfer Status

```typescript
import { getTransferStatus, getActiveTransfers } from "~/lib/rclone";

// Get status of a specific transfer
const status = await getTransferStatus("operation-id");
if (status) {
  console.log(`Status: ${status.status}, Progress: ${status.progress?.percentage}%`);
}

// Get all active transfers
const activeTransfers = getActiveTransfers();
console.log(`${activeTransfers.length} transfers in progress`);
```

### Cancelling Transfers

```typescript
import { cancelTransfer, cancelAllTransfers } from "~/lib/rclone";

// Cancel a specific transfer
const cancelled = await cancelTransfer("operation-id");

// Cancel all active transfers
const allCancelled = await cancelAllTransfers();
```

## TypeScript Interfaces

### TransferMetadata

```typescript
interface TransferMetadata {
  operationId: string;
  type: "file" | "folder" | "batch";
  status: "pending" | "running" | "completed" | "failed" | "cancelled";
  startTime: Date;
  endTime?: Date;
  sourceService: ServiceType;
  sourceAccountId: string;
  destService: ServiceType;
  destAccountId: string;
  error?: string;
  progress?: TransferProgress;
  data?: Record<string, unknown>;
}
```

### TransferProgress

```typescript
interface TransferProgress {
  percentage: number;
  bytesTransferred: number;
  totalBytes: number;
  transferRate?: number;
  estimatedTimeRemaining?: number;
  filesTransferred?: number;
  totalFiles?: number;
}
```

### TransferCallbacks

```typescript
interface TransferCallbacks {
  onStart?: (metadata: TransferMetadata) => void;
  onProgress?: (metadata: TransferMetadata) => void;
  onComplete?: (metadata: TransferMetadata) => void;
  onError?: (metadata: TransferMetadata, error: string) => void;
  onCancel?: (metadata: TransferMetadata) => void;
}
```

## Error Handling

The service uses the standardized `CloudStorageError` class with proper error codes:

```typescript
try {
  await transferFile(request);
} catch (error) {
  if (error instanceof CloudStorageError) {
    console.error(`Transfer failed: ${error.message} (${error.code})`);
    console.error(`Service: ${error.serviceType}, Account: ${error.accountId}`);
  }
}
```

## Integration with Existing Hooks

The service can be easily integrated with existing React hooks or used to create new ones:

```typescript
import { useCallback, useState } from "react";
import { transferFile, type FileTransferRequest, type TransferMetadata } from "~/lib/rclone";

export function useFileTransfer() {
  const [isLoading, setIsLoading] = useState(false);
  const [progress, setProgress] = useState<TransferMetadata | null>(null);
  const [error, setError] = useState<string | null>(null);

  const startTransfer = useCallback(async (request: FileTransferRequest) => {
    setIsLoading(true);
    setError(null);

    try {
      await transferFile(request, {
        onProgress: setProgress,
        onComplete: (metadata) => {
          setProgress(metadata);
          setIsLoading(false);
        },
        onError: (metadata, error) => {
          setError(error);
          setIsLoading(false);
        },
      });
    } catch (err) {
      setError(err instanceof Error ? err.message : "Transfer failed");
      setIsLoading(false);
    }
  }, []);

  return { startTransfer, isLoading, progress, error };
}
```