# Known RefgetStore Limitation: Case-Insensitive File System Conflicts ## Summary RefgetStore fails to properly store and retrieve sequences when running on case-insensitive file systems (like Dropbox-synced directories) due to sha512t24u digest prefixes differing only in case. ## Symptoms - `export_fasta()` fails with: "File not found locally and no remote source configured" - Some sequences are recorded in the index but missing from disk - Example: 610 sequences in index, but only 588 sequence files exist ## Root Cause RefgetStore uses the first 2 characters of sha512t24u digests as directory prefixes (e.g., `sequences/Lc/`). Base64url encoding produces case-sensitive characters, so directories like `0F` and `0f` can exist. On case-insensitive file systems (macOS default, Dropbox sync on Linux), these create conflicts: - `0F` directory is created first - `0f` directory cannot be created (or gets renamed to "0f (Case Conflict)") - Sequences with `0f` prefix never get written ## Evidence ``` $ ls sequences/ | grep -i "^0f" 0F 0f (Case Conflict) ``` Missing prefixes (19 total): `0f, 4w, 9n, Iq, Lc, Lq, Pj, Pq, Rc, Rr, Te, Tq, Uu, aw, dg, fu, iz, ji, vr` All correspond to Dropbox case conflicts, not gtars logic errors. ## Workarounds 1. Store RefgetStore databases on local disk (not Dropbox/iCloud synced directories) 2. Use a case-sensitive file system 3. Use longer prefixes (3+ chars) to reduce collision probability ## Potential Fixes 1. **Normalize to lowercase**: Use `prefix.lower()` for all directory names -- 2. **Use hex encoding**: Replace base64url with hex for prefixes (only 0-9, a-f) 3. **Detect and handle conflicts**: Check for case conflicts before writing ## Environment - gtars version: 0.5.2 - File system: Dropbox-synced directory on Linux (ext4, but Dropbox enforces case-insensitivity) - Input: HG002 pangenome FASTA (610 sequences) ## Additional insights: - it's not just the folder structure, but the sequence file names that are also case sensitive. - even if you solved this problem for folders (eg using `lower()`), it could lead to clashes in the .seq file paths. - so it could be dangerous to use on a case-insensitive filesystem just because it increases probability of clashes. - nevertheless, it might still work for smaller databases where clash risk is lower?