From 8926cf4000e87b27b21da1ba1e9db97b2d02ab23 Mon Sep 17 00:00:00 2001 From: Paul Jenkins Date: Thu, 26 Mar 2026 11:46:30 +0000 Subject: [PATCH] added prd for productisation --- README.md | 5 + docs/PRD.md | 480 +++++++++++++++++++++++++++++++++++++++++++++++++ docs/README.md | 3 + 3 files changed, 488 insertions(+) create mode 100644 docs/PRD.md create mode 100644 docs/README.md diff --git a/README.md b/README.md index 3de5f91..5882fbe 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,11 @@ A small web app with a Go backend and React frontend for listing, uploading, dow - `backend`: Go HTTP API for S3 operations - `frontend`: React + Vite client +- `docs`: product and implementation planning documents + +## Product Documentation + +- `docs/PRD.md`: product requirements document for the full application ## Backend configuration diff --git a/docs/PRD.md b/docs/PRD.md new file mode 100644 index 0000000..a219491 --- /dev/null +++ b/docs/PRD.md @@ -0,0 +1,480 @@ +# Product Requirements Document + +## Product + +**Name:** S3 Mover +**Document version:** 0.1 +**Date:** 2026-03-26 +**Status:** Draft + +## Overview + +S3 Mover is a web application for securely managing files across one or more S3-compatible buckets. The system provides a React frontend and Go backend, with SQLite as the system of record for users, bucket access, invitations, audit logs, and authentication-related metadata. + +The existing proof of concept demonstrated that a Go backend can enumerate files and perform upload, download, and delete operations against S3-compatible object storage. This product extends that proof of concept into a production-oriented multi-user application with user management, per-user bucket credentials, invitation-based onboarding, mandatory 2FA, audit logging, and Docker deployment. + +## Problem Statement + +The proof of concept is functionally viable, but it is not suitable for real users because: + +- There is no user management or role-based access control. +- Storage credentials are configured globally instead of per user and per bucket. +- There is no secure onboarding flow. +- There is no multi-factor authentication. +- There is no audit trail for file operations. +- Sensitive bucket configuration is not yet modeled as protected server-side data. + +The product must evolve into a secure internal application that allows administrators to manage users and user-specific bucket credentials, while allowing standard users to operate only on the buckets assigned to them. + +## Goals + +- Provide secure user authentication using email, password, and TOTP-based 2FA. +- Support administrator-led invitation onboarding via Resend email. +- Allow administrators to manage users and assign one or more bucket configurations to each user. +- Keep all sensitive bucket credentials server-side only. +- Provide file browsing and upload, download, and delete capabilities for the assigned bucket(s) of a logged-in user. +- Record auditable logs of bucket operations. +- Persist system data in SQLite. +- Package the application for Docker deployment. + +## Non-Goals + +- End-user self-registration. +- SSO, SAML, or OAuth identity providers in the first release. +- Bucket creation or infrastructure provisioning. +- Advanced IAM policy editing or credential rotation workflows. +- Object versioning, lifecycle management, or ACL management. +- Bulk file operations beyond single upload, download, and delete. +- Fine-grained per-prefix authorization within a bucket in the first release. + +## Users and Roles + +### Administrator + +The administrator manages the system and has access to all administrative functions. + +Capabilities: + +- Log in with email, password, and TOTP +- View all registered users +- View details for an individual user +- Invite a new user by email +- Delete a user account +- Add one or more bucket credential sets to a user +- Remove a bucket credential set from a user +- View audit logs for all file operations + +### Standard User + +The standard user interacts only with buckets assigned to their account. + +Capabilities: + +- Complete invitation onboarding +- Set a strong password +- Enroll a TOTP authenticator during first-time setup +- Log in with email, password, and TOTP +- View the list of assigned buckets in a left sidebar +- Select a bucket and view its contents +- Upload, download, and delete files within assigned buckets + +## Key User Journeys + +### 1. Administrator Invites a User + +1. Administrator logs into the application. +2. Administrator navigates to the user management page. +3. Administrator creates a new user by entering at least an email address. +4. System creates a pending user record and a time-limited invitation token. +5. System sends an invitation email through Resend. +6. Email contains a secure link to the onboarding page. + +### 2. User Accepts Invite and Creates Credentials + +1. User clicks the invitation link. +2. System validates the invitation token. +3. User is prompted to set a strong password. +4. User is required to enroll a TOTP authenticator before activation completes. +5. User confirms setup by entering a valid TOTP code. +6. System activates the account and marks the invitation as consumed. + +### 3. User Logs In + +1. User enters email address and password. +2. System validates credentials. +3. User is prompted for a TOTP code. +4. System issues a JWT after successful second-factor verification. +5. User is redirected to the main application. + +### 4. User Operates on a Bucket + +1. User logs in successfully. +2. User sees a sidebar listing assigned buckets. +3. User selects a bucket. +4. System displays the bucket name and endpoint at the top of the page. +5. System shows a paginated file list with 15 items per page. +6. User uploads, downloads, or deletes files. +7. System records an audit log entry for each file operation. + +### 5. Administrator Manages Bucket Credentials for a User + +1. Administrator navigates to a user detail page. +2. Administrator adds a bucket credential set for that user. +3. Administrator enters: + - AWS access key ID + - AWS secret access key + - AWS region + - bucket name + - S3-compatible endpoint +4. System stores those credentials securely in SQLite. +5. User gains access to that bucket after next authorized fetch or session refresh. +6. Administrator may remove a bucket configuration from the user later. + +## Functional Requirements + +## FR-1 Authentication + +- The system shall support login using email address and password. +- The system shall require TOTP as a second factor for all users. +- The system shall issue JWTs for authenticated API access. +- The system shall reject access to protected routes without a valid JWT. +- The system shall support logout by invalidating the client session. + +## FR-2 Invitation and Onboarding + +- Only an administrator may create a new user. +- Creating a user shall generate an invite email sent through Resend. +- The invite shall contain a secure, time-limited, single-use token. +- The invite flow shall require the user to set a strong password. +- The invite flow shall require TOTP setup before account activation. +- An invited user shall not be able to log in until onboarding is complete. + +## FR-3 User Management + +- The administrator shall have a page listing all users. +- The user list shall include at minimum email address, status, role, created date, and last login date if available. +- The administrator shall be able to click into a user detail page. +- The administrator shall be able to delete a user account. +- Deleting a user shall revoke future access immediately. + +## FR-4 Bucket Assignment + +- A user shall be assigned at least one bucket credential set to access storage. +- A user may have multiple bucket credential sets. +- Each bucket credential set shall be unique to the user and shall not be shared by design. +- The administrator shall be able to create, view metadata for, and remove bucket assignments for a user. +- Sensitive values such as access keys and secrets shall not be shown back to the browser after creation, except where explicitly required for a masked administrative UX. + +## FR-5 Bucket Browsing and File Operations + +- A logged-in user shall see assigned buckets in a left sidebar. +- Selecting a bucket shall show: + - bucket name + - endpoint + - paginated file listing +- Pagination shall be fixed at 15 items per page. +- The system shall support upload, download, and delete operations for files in the selected bucket. +- The system shall support S3-compatible providers, not just AWS S3. + +## FR-6 Audit Logging + +- The system shall log all user file operations. +- Each log entry shall include at minimum: + - timestamp + - user ID + - user email + - bucket identifier + - file key + - operation type +- The operation type shall support at minimum: + - upload + - download + - delete +- Only administrators shall be able to view logs. + +## FR-7 Administrative Access Control + +- Only administrators shall be able to: + - invite users + - view all users + - view user details + - manage user bucket assignments + - delete users + - view audit logs + +## FR-8 Deployment and Runtime + +- The application shall run in Docker. +- The backend shall be implemented in Go. +- The frontend shall be implemented in React. +- SQLite shall be the primary application database. + +## Security Requirements + +- Sensitive bucket credentials shall never be sent to the browser. +- Sensitive bucket credentials shall never be included in JWT payloads. +- JWTs shall be signed server-side with a configurable secret or private key. +- Passwords shall be stored only as strong salted hashes. +- TOTP secrets shall be stored securely server-side. +- Invite tokens shall be stored hashed or otherwise protected server-side and shall expire. +- The application shall enforce HTTPS in deployed environments. +- The application shall implement CSRF protections if JWTs are stored in cookies. +- The application shall implement rate limiting or equivalent brute-force protection for authentication endpoints. +- All sensitive configuration shall be provided through environment variables or secret injection at runtime. +- Administrative and user actions shall be authorized server-side for every request. + +## UX Requirements + +### Authentication + +- The application shall provide: + - login page + - TOTP challenge page + - invitation acceptance page + - password setup form + - TOTP enrollment page with QR code and manual key fallback + +### Main User Application + +- The main authenticated layout shall include: + - left sidebar for bucket selection + - top area showing selected bucket name and endpoint + - file listing view + - pagination controls + - upload action + - download action + - delete action + +### Administration + +- The administrator area shall include: + - user list page + - user detail page + - add bucket assignment flow + - audit log page + +## Data Model Requirements + +The following logical entities are required. + +### users + +- id +- email +- role +- status +- password_hash +- totp_secret +- totp_enabled +- invited_at +- activated_at +- last_login_at +- created_at +- updated_at + +### invitations + +- id +- user_id +- email +- token_hash +- expires_at +- consumed_at +- created_by_user_id +- created_at + +### user_bucket_credentials + +- id +- user_id +- display_name +- aws_access_key_id +- aws_secret_access_key +- aws_region +- s3_bucket +- aws_endpoint_url +- use_path_style +- created_at +- updated_at + +### audit_logs + +- id +- user_id +- user_email +- bucket_credential_id +- bucket_name +- endpoint +- file_key +- operation +- created_at +- request_ip +- user_agent + +### sessions or token metadata + +Implementation may use stateless JWTs only, but the design should allow optional storage for token revocation, refresh sessions, or login tracking if needed. + +## Recommended Status Values + +### User status + +- invited +- active +- disabled + +### Role + +- admin +- user + +### Audit operation + +- upload +- download +- delete + +## API Requirements + +The final API shape may change, but the following resources are expected. + +### Authentication + +- `POST /api/auth/login` +- `POST /api/auth/totp/verify` +- `POST /api/auth/logout` +- `GET /api/auth/me` + +### Invitations + +- `POST /api/admin/users/invite` +- `GET /api/invitations/:token` +- `POST /api/invitations/:token/accept` + +### Administration + +- `GET /api/admin/users` +- `GET /api/admin/users/:userId` +- `DELETE /api/admin/users/:userId` +- `POST /api/admin/users/:userId/buckets` +- `DELETE /api/admin/users/:userId/buckets/:bucketCredentialId` +- `GET /api/admin/audit-logs` + +### User bucket access + +- `GET /api/me/buckets` +- `GET /api/me/buckets/:bucketCredentialId/files?page=1&pageSize=15` +- `POST /api/me/buckets/:bucketCredentialId/files` +- `GET /api/me/buckets/:bucketCredentialId/files/:key` +- `DELETE /api/me/buckets/:bucketCredentialId/files/:key` + +## Business Rules + +- Every non-admin user must authenticate with password plus TOTP. +- An invitation may only be used once. +- A bucket credential set belongs to exactly one user. +- A user may have zero bucket assignments during setup or after administrative changes, but a user must have at least one assignment to perform storage operations. +- The server shall resolve storage access from the selected bucket credential ID and not from any client-supplied raw credential values. +- The browser may know a bucket display name, bucket name, and endpoint, but not secret credentials. + +## Technical Requirements + +### Backend + +- Go HTTP API +- SQLite persistence +- JWT authentication middleware +- Password hashing using a modern algorithm such as Argon2id or bcrypt +- TOTP support using a standard authenticator-compatible implementation +- Resend integration for invitation emails +- S3-compatible client abstraction for storage operations +- Audit log persistence for all storage mutations and downloads + +### Frontend + +- React application +- Authenticated routing +- Admin-only views and user views +- Sidebar bucket navigation +- Paginated bucket file list with page size fixed to 15 +- Forms for onboarding, login, TOTP verification, user management, and bucket assignment + +### Infrastructure and Deployment + +- Single Docker image or Docker Compose-based local runtime +- Persistent storage volume for SQLite database +- Environment-based configuration for JWT secret, Resend API key, and bootstrap admin settings + +## Reporting and Logging Requirements + +- Application logs shall capture authentication failures, invitation events, administrative changes, and storage operation failures. +- Audit logs shall be queryable by administrators. +- Audit log views should support filtering by: + - date range + - user + - bucket + - operation + +## Constraints + +- SQLite is the required datastore for the initial production-ready version. +- The frontend must not directly communicate with S3-compatible storage using raw bucket credentials. +- All storage operations must be proxied through the Go backend. +- Sensitive settings must remain server-side. + +## Assumptions + +- The first release supports one administrator bootstrap path via environment configuration, migration seed, or initial setup command. +- Invitation emails will be transactional emails sent through Resend. +- TOTP apps such as 1Password, Google Authenticator, or Authy are acceptable authenticators. +- The application is intended for trusted organizational users rather than open public sign-up. + +## Open Questions + +- How should the first administrator account be bootstrapped? +- Should administrators also be able to perform bucket operations, or only manage users and logs? +- Should deleted users be hard-deleted or soft-deleted for audit retention? +- Should bucket secrets be encrypted at rest in SQLite using an application-level encryption key? +- Should JWTs be short-lived access tokens only, or should the system also support refresh tokens? +- What are the password strength rules beyond “strong password”? +- What is the invite expiration window? +- Should audit logs include failed file operations in addition to successful ones? +- Is account disablement required in addition to deletion? +- Are per-user bucket display names needed in the UI, or is raw bucket name sufficient? + +## Success Criteria + +- Administrators can invite, onboard, and manage users without manual database changes. +- Standard users can access only their assigned buckets. +- Bucket credentials never appear in browser network payloads or rendered UI. +- Users must complete TOTP enrollment before account activation. +- All file uploads, downloads, and deletes are auditable by administrators. +- The application runs successfully in Docker with SQLite persistence. + +## Proposed Delivery Phases + +### Phase 1: Identity and Administration Foundation + +- SQLite schema and migrations +- bootstrap administrator +- login, password hashing, JWT issuance +- TOTP enrollment and verification +- invite flow with Resend +- admin user list and user detail pages + +### Phase 2: Bucket Assignment and Secure Access + +- per-user bucket credential storage +- admin create and remove bucket credentials +- authenticated bucket list API +- user sidebar and selected-bucket view +- secure server-side storage operations using assigned credentials + +### Phase 3: Auditability and Production Hardening + +- audit log persistence and admin log viewer +- Docker packaging +- operational logging and error handling +- rate limiting and security hardening +- UX polish and edge-case handling + diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..b122737 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,3 @@ +# Documentation + +- [PRD](./PRD.md)