Refactor download architecture to be protocol-agnostic#124
Open
Refactor download architecture to be protocol-agnostic#124
Conversation
- Remove HTTP-specific fields (etag, lastModified, acceptRanges) from TaskRecord; all source-specific state now lives in sourceResumeState - Add buildResumeState(ResolvedSource, Long) to DownloadSource interface so each source manages its own resume state - Remove all HttpDownloadSource references from DownloadExecution (protocol-agnostic) - Rename headers→properties in DownloadSource.resolve() and KetchApi.resolve() - Update FileNameResolver to accept ResolvedSource instead of ServerInfo - Update SQLite schema to persist sourceType and sourceResumeState instead of HTTP-specific columns - Implement buildResumeState() in HttpDownloadSource, FtpDownloadSource, and TorrentDownloadSource https://claude.ai/code/session_016RRifFe1AwMyjU8NYHrEqC
- Extract shared download loop (downloadSegments/downloadBatch) into SegmentedDownloadHelper, eliminating ~200 lines of duplicated code between HttpDownloadSource and FtpDownloadSource - Both sources now delegate to SegmentedDownloadHelper.downloadAll() with a protocol-specific segment download lambda - Fix headers→properties rename across all KetchApi implementations: RemoteKetch, InstanceManager, FakeKetchApi, ResolveUrlRequest, ServerRoutes - Update all test fake DownloadSource objects: rename headers→properties, add buildResumeState() implementation - Update DefaultFileNameResolverTest to use ResolvedSource instead of ServerInfo - Remove obsolete TaskRecordTest for removed HTTP-specific fields - Update DownloadContext KDoc to be protocol-agnostic https://claude.ai/code/session_016RRifFe1AwMyjU8NYHrEqC
Add updateResumeState() to DownloadSource interface so sources can persist resume state incrementally during download. This completes the source-managed resume state lifecycle (Step 7 of the refactoring plan). - DownloadSource: new suspend fun updateResumeState() with default null - DownloadExecution: save job now calls updateResumeState() and persists the result alongside segment snapshots - TorrentDownloadSource: implements updateResumeState() to persist bitfield/resume data via activeSession, replacing the dead code in monitorProgress() that built resume state but never persisted it https://claude.ai/code/session_016RRifFe1AwMyjU8NYHrEqC
Documents the 7-step plan for decoupling the download architecture from HTTP-specific assumptions, enabling clean support for BitTorrent, HLS, and future protocol sources. https://claude.ai/code/session_016RRifFe1AwMyjU8NYHrEqC
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
Test Results340 tests - 705 340 ✅ - 705 1s ⏱️ -13s Results for commit 9a4bd6c. ± Comparison against base commit ea01275. This pull request removes 705 tests.♻️ This comment has been updated with latest results. |
…oad-architecture-fV3x2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR refactors the core download engine to remove HTTP-specific assumptions and abstractions, enabling support for fundamentally different protocols like BitTorrent that don't fit the byte-range segment model.
Key Changes
Removed HTTP-specific fields from TaskRecord:
etag,lastModified, andacceptRangesare now stored in the protocol-agnosticsourceResumeStatefield instead of being baked into the persistent record.Eliminated HttpDownloadSource references from DownloadExecution: Added
buildResumeState()andupdateResumeState()methods to theDownloadSourceinterface so each source manages its own resume state lifecycle. DownloadExecution no longer directly references HTTP-specific metadata keys.Extracted shared segmented download logic: Created
SegmentedDownloadHelperto consolidate the nearly identical download loop, progress aggregation, and connection-change watcher logic that was duplicated between HTTP and FTP sources. Both sources now delegate to this helper instead of reimplementing the same logic.Renamed
headersparameter toproperties: ChangedDownloadSource.resolve()signature fromheaders: Map<String, String>toproperties: Map<String, String>to reflect that non-HTTP sources interpret this parameter differently (or ignore it entirely).Updated FileNameResolver to use ResolvedSource: Changed the interface to accept
ResolvedSourceinstead of HTTP-specificServerInfo, decoupling filename resolution from HTTP concerns.Enhanced TorrentDownloadSource: Added
buildResumeState()andupdateResumeState()implementations to persist torrent-specific state (info hash, resume data, selected files) and track the active session for incremental updates.Implementation Details
The
SegmentedDownloadHelperaccepts a genericdownloadSegmentlambda, allowing HTTP sources to useSegmentDownloaderwith HTTP Range requests while FTP sources provide their own FTP-specific download logic.Resume state is now always populated via
source.buildResumeState()during fresh downloads, eliminating the need for fallback logic in DownloadExecution.The refactoring maintains backward compatibility with existing
Segment-based progress reporting while allowing future protocols (like BitTorrent) to map their native concepts (pieces, bitfields) to the segment abstraction.All protocol-specific metadata is now stored opaquely in
SourceResumeState.data(JSON), making the core engine truly protocol-agnostic.https://claude.ai/code/session_016RRifFe1AwMyjU8NYHrEqC