--- title: Storage Package description: A Laravel-style file storage abstraction supporting local filesystem and cloud storage (S3) with a unified API. --- # Storage Package A Laravel-style file storage abstraction supporting local filesystem and cloud storage (S3) with a unified API. ## Installation ```bash bun add @stacksjs/storage ``` ## Basic Usage ```typescript import { Storage } from '@stacksjs/storage' // Write a file await Storage.put('documents/report.txt', 'Report content') // Read a file const content = await Storage.get('documents/report.txt') // Check if file exists const exists = await Storage.exists('documents/report.txt') // Delete a file await Storage.delete('documents/report.txt') ``` ## Configuration Configure storage in `config/filesystems.ts`: ```typescript export default { // Default disk driver: 'local', // Root directory for local storage root: process.cwd(), // Default file visibility defaultVisibility: 'private', // S3 configuration s3: { bucket: process.env.AWS_BUCKET, region: process.env.AWS_DEFAULT_REGION || 'us-east-1', prefix: '', // Optional path prefix endpoint: process.env.AWS_ENDPOINT, // For S3-compatible services credentials: { accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, }, }, // Public URL configuration publicUrl: { domain: process.env.ASSET_URL, }, } ``` ## Working with Disks ### Available Disks By default, three disks are configured: - `local`: private storage under `storage/app`, created on first write - `public`: web-accessible storage under `public`, served at `/storage` - `s3`: Amazon S3 or a compatible service - only registered when `AWS_S3_BUCKET` is set `local` is the default. Add your own by merging into `disks` from `config/filesystems.ts`, then augment the `KnownDisks` interface so `Storage.disk('…')` autocompletes it. ### Switching Disks ```typescript import { Storage } from '@stacksjs/storage' // Use local disk (default) await Storage.put('file.txt', 'content') // Use public disk await Storage.disk('public').put('images/logo.png', imageBuffer) // Use S3 disk await Storage.disk('s3').put('backups/db.sql', sqlDump) ``` ### Configuring Custom Disks ```typescript import { Storage } from '@stacksjs/storage' // Add a custom disk at runtime Storage.configure('backups', { driver: 'local', root: '/mnt/backups', visibility: 'private', }) // Use custom disk await Storage.disk('backups').put('daily.tar.gz', archiveData) ``` ## File Operations ### Writing Files ```typescript import { Storage } from '@stacksjs/storage' // Write string content await Storage.put('documents/readme.txt', 'Hello World') // Write buffer const imageBuffer = await fetch('https://example.com/image.png') .then(r => r.arrayBuffer()) await Storage.put('images/photo.png', new Uint8Array(imageBuffer)) // Write with visibility await Storage.disk('s3').put('reports/q1.pdf', pdfData, { visibility: 'public' }) ``` ### Reading Files ```typescript import { Storage } from '@stacksjs/storage' // Read as string const content = await Storage.get('documents/readme.txt') // Read as buffer const disk = Storage.disk('local') const buffer = await disk.readToBuffer('images/photo.png') // Stream large files const stream = Storage.disk('s3').readStream('videos/large.mp4') ``` ### Checking File Existence ```typescript import { Storage } from '@stacksjs/storage' // Check if file exists const exists = await Storage.exists('documents/readme.txt') // Check if file is missing const missing = await Storage.missing('documents/readme.txt') if (missing) { await Storage.put('documents/readme.txt', 'Default content') } ``` ### Deleting Files ```typescript import { Storage } from '@stacksjs/storage' // Delete single file await Storage.delete('temp/cache.txt') // Delete multiple files await Storage.disk('local').deleteFile('file1.txt') await Storage.disk('local').deleteFile('file2.txt') ``` ### Copying and Moving Files ```typescript import { Storage } from '@stacksjs/storage' // Copy file await Storage.copy('original.txt', 'backup/original.txt') // Move file await Storage.move('uploads/temp.txt', 'documents/final.txt') ``` ## File Information ### Getting File Metadata ```typescript import { Storage } from '@stacksjs/storage' // Get file size in bytes const size = await Storage.size('documents/report.pdf') // Get last modified timestamp const lastModified = await Storage.lastModified('documents/report.pdf') // Get MIME type const mimeType = await Storage.mimeType('images/photo.jpg') // Returns: 'image/jpeg' // Get file checksum const md5 = await Storage.checksum('documents/report.pdf', 'md5') const sha256 = await Storage.checksum('documents/report.pdf', 'sha256') ``` ### Getting Public URLs ```typescript import { Storage } from '@stacksjs/storage' // Get public URL for a file const url = await Storage.url('images/logo.png') // For S3, get temporary signed URL const disk = Storage.disk('s3') const signedUrl = await disk.temporaryUrl('private/document.pdf', 3600) // URL expires in 1 hour ``` ## Directory Operations ### Creating Directories ```typescript import { Storage } from '@stacksjs/storage' // Create directory await Storage.makeDirectory('uploads/2024/01') // Nested directories are created automatically await Storage.put('deep/nested/path/file.txt', 'content') ``` ### Listing Files ```typescript import { Storage } from '@stacksjs/storage' // List files in directory (shallow) for await (const item of Storage.files('documents')) { console.log(item.path, item.type) // 'file' or 'directory' } // List all files recursively (deep) for await (const item of Storage.allFiles('documents')) { if (item.type === 'file') { console.log(item.path) } } ``` ### Deleting Directories ```typescript import { Storage } from '@stacksjs/storage' // Delete directory and all contents await Storage.deleteDirectory('temp') ``` ## File Uploads ### Handling Uploaded Files ```typescript import { route } from '@stacksjs/router' route.post('/upload', async (req) => { // Get uploaded file const file = req.file('document') if (file) { // Store with auto-generated name const path = await file.store('uploads') // Returns: 'uploads/abc123.pdf' // Store with custom name const customPath = await file.storeAs('uploads', 'report-2024.pdf') // Returns: 'uploads/report-2024.pdf' // Store on specific disk await file.store('uploads', { disk: 's3' }) } return Response.json({ success: true }) }) ``` ### UploadedFile Methods ```typescript import { UploadedFile } from '@stacksjs/storage' route.post('/upload', async (req) => { const file = req.file('avatar') if (file) { // Get file information const originalName = file.getClientOriginalName() const extension = file.getClientOriginalExtension() const mimeType = file.getMimeType() const size = file.getSize() // Validate file if (!['image/jpeg', 'image/png'].includes(mimeType)) { return Response.json({ error: 'Invalid file type' }, { status: 400 }) } if (size > 5 _ 1024 _ 1024) { // 5MB return Response.json({ error: 'File too large' }, { status: 400 }) } // Store file await file.store('avatars') } }) ``` ### Multiple File Uploads ```typescript route.post('/upload-multiple', async (req) => { const files = req.getFiles('documents') const paths = [] for (const file of files) { const path = await file.store('documents') paths.push(path) } return Response.json({ uploaded: paths }) }) ``` ## S3 Storage ### Basic S3 Operations ```typescript import { Storage } from '@stacksjs/storage' const s3 = Storage.disk('s3') // Upload to S3 await s3.put('uploads/file.txt', 'content') // Download from S3 const content = await s3.get('uploads/file.txt') // Delete from S3 await s3.delete('uploads/file.txt') ``` ### S3 with Custom Endpoint For S3-compatible services like MinIO, DigitalOcean Spaces, or Cloudflare R2: ```typescript // config/filesystems.ts export default { s3: { bucket: 'my-bucket', region: 'us-east-1', endpoint: 'https://s3.example.com', // Custom endpoint credentials: { accessKeyId: process.env.S3_ACCESS_KEY, secretAccessKey: process.env.S3_SECRET_KEY, }, }, } ``` ### Provider presets Most object stores speak the S3 API, so they are `s3` disks with a different endpoint. Each has a helper that fills in the endpoint and region so you do not have to look them up: ```typescript import { backblazeDisk, filebaseDisk, gcsDisk, hetznerDisk, r2Disk, s3Disk } from '@stacksjs/storage' r2Disk('assets', process.env.R2_ACCOUNT_ID!) // Cloudflare R2 gcsDisk('assets') // Google Cloud Storage (interop mode) filebaseDisk('assets') // Filebase (IPFS-backed) backblazeDisk('assets', 'us-west-004') // Backblaze B2 hetznerDisk('assets', 'fsn1') // Hetzner Object Storage ``` Two of them need something extra: - **R2** serves public objects only from a mapped custom domain or a `pub-.r2.dev` address, never from its API host. Set `url` to that domain, or `publicUrl()` returns a well-formed address that does not resolve to the object. - **GCS** authenticates with HMAC keys from Cloud Storage > Settings > Interoperability, not with a service-account JSON file. The interop API covers object CRUD, listing and signed URLs; GCS-only features such as resumable-upload sessions have no S3 verb and are not reachable through it. ### Azure Blob Storage Azure is the one provider here with no S3-compatible API, so it has its own driver rather than an endpoint on the `s3` one: ```typescript import { azureDisk, Storage } from '@stacksjs/storage' // config/filesystems.ts export default { disks: { azure: azureDisk('uploads', process.env.AZURE_STORAGE_ACCOUNT!, { accountKey: process.env.AZURE_STORAGE_ACCOUNT_KEY, url: 'https://cdn.example.com', // optional: a CDN or custom domain }), }, } await Storage.disk('azure').put('reports/q1.pdf', bytes) const url = await Storage.disk('azure').signedUrl('reports/q1.pdf', { expiresIn: 3600 }) ``` **Setup.** Create a storage account and a container in the Azure portal, then take a key from Access keys. `accountKey` is what authorizes server-side calls and what mints signed URLs; a pre-minted `sasToken` works instead for a disk that only reads, but signed URLs are then unavailable, because re-serving that token would hand out its whole grant. Three things behave differently from an S3 disk: - **Visibility is a property of the container, not the blob.** Azure has no per-object ACL, so `changeVisibility()` refuses rather than silently doing nothing. `visibility()` reports the container's public access level, which is every blob's answer. A per-object grant on Azure is a SAS - that is `signedUrl()`. - **Large uploads are blocked, not multipart.** `putStream` stages blocks and commits them in one Put Block List, so a reader mid-upload sees the previous version or nothing, never a partial one. Abandoned blocks are garbage collected by Azure after a week rather than billed indefinitely. - **The emulator scopes the account by path.** Point `endpoint` at `http://127.0.0.1:10000/devstoreaccount1` to run against Azurite. ### File manager metadata and processing The dashboard's file manager has two layers, and the split is worth knowing before extending it. **Storage operations** map to a `StorageAdapter` method and go straight to the disk: list, upload, create folder, rename, visibility, duplicate, delete. **Metadata** has nowhere to live on a disk. Favourites and tags are in `storage_items` keyed by `(disk, path)`; background processing is in `storage_item_tasks` keyed by `(disk, path, kind)`. The listing comes from the disk and both are joined onto it, so the disk stays authoritative and the tables annotate it (stacksjs/stacks#2577, #2578). ``` PUT /api/dashboard/files/favorite { disk, path, favorite } PUT /api/dashboard/files/tags { disk, path, tags: [...] } POST /api/dashboard/files/reprocess { disk, path, kinds?: [...], profile? } ``` An upload dispatches the work its content type calls for - image variants via `ts-images`, mp4 and HLS via `ts-videos`, tags from a vision model - and writes the derivatives back to the same disk under `.variants//`, hidden from the listing. Nothing runs inside the request: a transcode is minutes, and a vision call is a round trip to a third party. ### File previews Every upload the file manager can show more than an icon for gets a `preview` task (stacksjs/stacks#308). `GenerateStoragePreviewJob` draws it once, on the `media` queue, and stores it as an ordinary file beside the other derivatives: ``` .variants//preview.png # images, WAV waveforms, TrueType specimens .variants//preview.txt # text: the first 12 lines, 80 columns each ``` Previews are generated once and served as stored files, not rendered on demand at the edge, so they reuse the task row, the queue and the reconciliation the rest of the pipeline already has. A rename through the dashboard moves the `.variants/` folder with the file, and a delete removes it. Only what the framework's own libraries can draw is drawn. Nothing shells out to ffmpeg, ImageMagick or a headless browser: | Kind | Preview | How | |---|---|---| | PNG, JPEG, GIF, BMP, WebP, AVIF | PNG, longest edge 320px, never upscaled | `ts-images` decode, resize, PNG encode, all in memory | | WAV (uncompressed PCM) | 320x120 waveform | peaks read straight off the samples, drawn with `ts-images` | | TrueType `.ttf` | 320x200 specimen | the font itself, through the `ts-images` glyph rasterizer | | Text, Markdown, CSV, JSON, YAML, code | plain-text snippet | first lines of the file, control characters stripped | Everything else with a preview kind is recorded as a `skipped` task, with the reason in `error`, and is never queued: | Kind | Why it is skipped | |---|---| | SVG | Never rasterized: it is markup, and drawing attacker-supplied SVG is script execution | | Video | Decoding a frame goes through WebCodecs, which Bun does not provide | | MP3, AAC, FLAC, Ogg | Same: `@ts-audio` decodes through WebCodecs | | PDF | There is no homegrown PDF renderer | | OTF (CFF), WOFF, WOFF2 | The rasterizer reads TrueType outlines only | | HEIC, TIFF | No decoder for either | Archives and unknown binaries get no preview row at all. The job checks the bytes before drawing. A file whose bytes disagree with its name - a PDF called `.png`, a WOFF called `.ttf`, NUL bytes in a `.txt` - is skipped rather than drawn as whatever it turned out to be. A file that is what it claims and is broken fails, counts the attempt, and is retried by the queue. `skipped` is terminal and does not count towards a file's `processing` state, so a photo whose preview was skipped and whose variants finished reads as `done`. The listing carries the result. `preview` is the URL of the stored image on a public disk, and `previewText` is the snippet on any disk, because the server reads it. A private disk has no URL for its originals, so it gets none for their previews either. Re-run one with: ``` POST /api/dashboard/files/reprocess { disk, path, kinds: ['preview'] } ``` Outside the dashboard, `generateStoragePreview(adapter, store, disk, path)` in `app/Actions/Dashboard/Content/file-preview.ts` is what the job runs, and `previewPathFor(path)` is where it writes. ### S3 Visibility ```typescript import { Storage } from '@stacksjs/storage' const s3 = Storage.disk('s3') // Upload as public await s3.put('public/image.jpg', imageData, { visibility: 'public' }) // Upload as private (default) await s3.put('private/document.pdf', pdfData) // Get visibility const visibility = await s3.getVisibility('public/image.jpg') // Returns: 'public' or 'private' // Change visibility await s3.setVisibility('file.txt', 'public') ``` ## Storage Adapters ### Using Adapters Directly ```typescript import { createLocalStorage, S3StorageAdapter } from '@stacksjs/storage' import { S3Client } from '@aws-sdk/client-s3' // Create local storage adapter const local = createLocalStorage({ root: '/data/storage' }) await local.write('file.txt', 'content') const content = await local.readToString('file.txt') // Create S3 adapter const s3Client = new S3Client({ region: 'us-east-1' }) const s3 = new S3StorageAdapter(s3Client, { bucket: 'my-bucket', region: 'us-east-1', prefix: 'app/' }) ``` ### Adapter Methods ```typescript // Common adapter methods await adapter.write(path, contents) await adapter.readToString(path) await adapter.readToBuffer(path) await adapter.fileExists(path) await adapter.deleteFile(path) await adapter.copyFile(from, to) await adapter.moveFile(from, to) await adapter.fileSize(path) await adapter.lastModified(path) await adapter.mimeType(path) await adapter.checksum(path, { algorithm: 'md5' }) await adapter.publicUrl(path) await adapter.createDirectory(path) await adapter.deleteDirectory(path) adapter.list(path, { deep: true }) ``` ## Utility Functions ### File System Helpers ```typescript import { copyFile, deleteFile, deleteFolder, ensureDir, fileExists, folderExists, getFileHash, getFileSize, glob, readFile, writeFile, zipFolder, unzipFile } from '@stacksjs/storage' // Check existence const exists = await fileExists('/path/to/file.txt') const dirExists = await folderExists('/path/to/dir') // Read/Write const content = await readFile('/path/to/file.txt') await writeFile('/path/to/file.txt', 'content') // Copy/Delete await copyFile('/source.txt', '/dest.txt') await deleteFile('/file.txt') await deleteFolder('/directory') // Ensure directory exists await ensureDir('/path/to/directory') // File info const size = await getFileSize('/path/to/file.txt') const hash = await getFileHash('/path/to/file.txt', 'sha256') // Glob patterns const files = await glob('**/*.ts', { cwd: '/src' }) // Compression await zipFolder('/source/dir', '/archive.zip') await unzipFile('/archive.zip', '/destination') ``` ## Storage Manager ### Manager Methods ```typescript import { Storage, StorageManager } from '@stacksjs/storage' // Get configured disks const disks = Storage.getConfiguredDisks() // Returns: ['local', 'public', 's3'] // Get default disk name const defaultDisk = Storage.getDefaultDisk() // Returns: 'local' // Set default disk Storage.setDefaultDisk('s3') // Get disk configuration const config = Storage.getDiskConfig('s3') // Reset manager (useful for testing) Storage.reset() ``` ## Edge Cases ### Handling Large Files ```typescript import { Storage } from '@stacksjs/storage' // For large files, use streaming const disk = Storage.disk('s3') // Read stream const readStream = disk.readStream('large-file.zip') // Write in chunks const largeData = await fetchLargeData() await disk.writeStream('output.dat', largeData) ``` ### Handling Missing Files ```typescript import { Storage } from '@stacksjs/storage' try { const content = await Storage.get('nonexistent.txt') } catch (error) { if (error.code === 'ENOENT') { console.log('File not found') } } // Or use exists check if (await Storage.exists('file.txt')) { const content = await Storage.get('file.txt') } ``` ### Handling Concurrent Access ```typescript import { Storage } from '@stacksjs/storage' import { CacheLock } from '@stacksjs/cache' async function updateFile(path: string, updater: (content: string) => string) { const lock = new CacheLock(cache, { lockTimeout: 5000 }) await lock.withLock(`file:${path}`, async () => { const content = await Storage.get(path) const updated = updater(content) await Storage.put(path, updated) }) } ``` ### Handling Special Characters in Paths ```typescript import { Storage } from '@stacksjs/storage' // Paths are automatically normalized await Storage.put('docs/file with spaces.txt', 'content') await Storage.put('docs/file-with-unicode-\u00e9.txt', 'content') // Use URL encoding for problematic characters const safePath = encodeURIComponent('file#with#hashes.txt') await Storage.put(`docs/${safePath}`, 'content') ``` ## API Reference ### Storage Facade Methods | Method | Description | |--------|-------------| | `disk(name?)` | Get disk instance | | `put(path, contents)` | Write file | | `get(path)` | Read file as string | | `exists(path)` | Check if file exists | | `missing(path)` | Check if file is missing | | `delete(path)` | Delete file | | `copy(from, to)` | Copy file | | `move(from, to)` | Move file | | `url(path)` | Get public URL | | `size(path)` | Get file size | | `lastModified(path)` | Get modification time | | `mimeType(path)` | Get MIME type | | `checksum(path, algo?)` | Get file checksum | | `makeDirectory(path)` | Create directory | | `deleteDirectory(path)` | Delete directory | | `files(path)` | List files (shallow) | | `allFiles(path)` | List files (deep) | ### Configuration Options | Option | Type | Description | |--------|------|-------------| | `driver` | string | Default disk driver | | `root` | string | Root directory | | `defaultVisibility` | string | Default file visibility | | `s3.bucket` | string | S3 bucket name | | `s3.region` | string | AWS region | | `s3.endpoint` | string | Custom S3 endpoint | | `s3.prefix` | string | Path prefix | | `s3.credentials` | object | AWS credentials |