Project
pdfbox
activeServerless multi-tenant file storage on AWS, all in Terraform. Presigned uploads, event-driven confirmation and idempotent writes.
go · aws · terraform · s3 · sqs · dynamodb
What it is
Serverless multi-tenant PDF storage on AWS, written in Go and provisioned with Terraform, where file bytes never pass through the backend.
The API writes metadata to DynamoDB and hands the browser a short-lived S3 presigned URL; the browser then uploads or downloads directly against S3. Lambda handles kilobytes of JSON whether the user uploads 12 KB or 50 MB.
Architecture
Lambda (Go) behind an API Gateway HTTP API, DynamoDB for metadata, S3 for objects, S3 notifications into SQS with a DLQ, IAM roles and JWT (HS256) for identity, all provisioned with Terraform.
The upload flow:
- The client calls
POST /fileswith filename, size and MIME type. - The API writes a
pendingrecord keyed by the JWT’suser_idand returns a presignedPUTURL, valid for 15 minutes, withContent-Typeand the exactContent-Lengthin the signature. - The browser uploads straight to S3.
- S3 emits
s3:ObjectCreated:*to an SQS queue. - A worker Lambda checks the object’s real size against the record and the 50 MB ceiling, then marks the record
uploaded, orrejectedand deletes the object.
What I learned
The README’s design decisions, in short:
- Presigned URLs instead of proxying. Proxying through Lambda would cap uploads at Lambda’s 6 MB request payload limit, well under the 50 MB the product allows.
- The confirmation comes from S3, not the client. The client can close the tab, lose connectivity or confirm without uploading anything; only S3 knows whether the object exists.
- Idempotency, because SQS is at-least-once. The worker treats a duplicate event as success, not as an error.
- Conditional writes instead of read-then-write. Every DynamoDB write carries a
ConditionExpression, so the check is atomic with the write. - Authorization is structural. The JWT’s user claim builds the partition key, so another user’s data is never in hand to be checked.
- No dual-write problem. The backend writes only to DynamoDB; S3 publishes the event atomically with the object write.
What’s next
From the README’s “Not yet implemented” list:
- The reconciliation Lambda and its EventBridge Scheduler rule (
cmd/event-bridgeexists but is empty). - The DynamoDB TTL that purges trashed records after a retention window.
- A restore-from-trash route.