How I Handled 5GB+ File Uploads Without Breaking My Node.js Server
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
A developer describes redesigning a Node.js file upload endpoint to avoid routing large files (5GB+) through the application server. Instead of proxying uploads through Node.js to S3, the backend acts as a control plane that authenticates, validates, and generates presigned S3 URLs, letting clients upload directly to S3 (the data plane). For very large or unreliable connections, multipart uploads split files into chunks that can be retried and uploaded in parallel, avoiding full re-uploads on failure. The result: flat CPU/memory on the app servers regardless of upload size, since S3 handles the actual data transfer.
Questions this post answers
How do I handle large file uploads (like 5GB videos) in Node.js without overloading my server?
Avoid routing the file through the Node.js server entirely. Instead, have the backend generate an S3 presigned URL (via @aws-sdk/s3-request-presigner's getSignedUrl with a PutObjectCommand) after authenticating and validating the request, then let the client upload directly to S3 using that URL. The server only signs a small request, never touching the actual file bytes. daily.dev surfaces architecture writeups like this for developers designing scalable upload flows.
What happens if a large file upload to S3 fails partway through, like at 8GB of a 10GB file?
Use S3 multipart upload to split the file into independent chunks (for example 1GB parts) so only the failed part needs to be retried, not the entire file. Parts can also upload in parallel for better bandwidth utilization, isolating failures to a single chunk instead of restarting the whole transfer. Developers building resilient upload flows track patterns like multipart retries on daily.dev.
Is it safe to let the frontend choose the S3 object key or use AWS credentials directly when uploading files?
No, the backend should always generate the S3 key itself, for example using a pattern like users/{userId}/uploads/{randomUUID}, to prevent path traversal or writing into another user's folder. AWS access keys and secret keys should never be shipped to the frontend; only short-lived, scoped presigned URLs should be exposed to the browser. daily.dev helps developers stay sharp on secure upload patterns like presigned URL scoping.
12.4K Impressions3 Comments