trustos/docs/DEPLOYMENT.md

TrustOS Deployment Guide

This guide provides comprehensive instructions for deploying TrustOS to production environments.

---

Table of Contents

• Overview
• Deployment Options
• Pre-Deployment Checklist
• Option 1: Railway Deployment
• Option 2: Render Deployment
• Option 3: VPS Deployment
• Option 4: Kubernetes Deployment
• Post-Deployment Steps
• Monitoring & Maintenance
• Backup & Recovery
• Troubleshooting

---

Overview

TrustOS can be deployed to various platforms depending on your needs and expertise:

• Railway: Easiest option, fully managed, good for quick deployment
• Render: Simple managed platform with good PostgreSQL support
• VPS: Full control, cost-effective for larger deployments
• Kubernetes: For enterprise-scale deployments with high availability

---

Deployment Options

Deployment Decision Tree

graph TD
    Start[Start Deployment] --> Budget{Budget?}
    Budget -->|< $50/mo| VPS[VPS Deployment]
    Budget -->|$50-200/mo| Managed{Managed Platform?}
    Budget -->|> $200/mo| K8s[Kubernetes]
    
    Managed -->|Yes| Railway{Need Simple?}
    Managed -->|No| Render[Render Deployment]
    
    Railway -->|Yes| RailwayDeploy[Railway Deployment]
    Railway -->|No| Render
    
    VPS --> VPSDeploy[VPS Deployment Guide]
    K8s --> K8sDeploy[Kubernetes Deployment]
    RailwayDeploy --> Done[Deployment Complete]
    Render --> Done
    VPSDeploy --> Done
    K8sDeploy --> Done
    
    style Start fill:#e8f5e9
    style Done fill:#e8f5e9
    style VPS fill:#fff3e0
    style Railway fill:#e3f2fd
    style Render fill:#f3e5f5
    style K8s fill:#fce4ec

Comparison

Platform	Difficulty	Cost	Control	Scalability	Best For
Railway	Easy	$$	Low	Medium	Quick MVP, small teams
Render	Easy	$$	Low	Medium	Simple apps, good Postgres
VPS	Medium	$	High	High	Cost-effective, custom needs
Kubernetes	Hard	$$$	High	Very High	Enterprise, high availability

---

Pre-Deployment Checklist

Before deploying to production, complete these steps:

1. Security Configuration

• Generate a secure SECRET_KEY using openssl rand -hex 32
• Set strong database passwords (minimum 16 characters, mixed case, numbers, symbols)
• Configure AI API keys with appropriate rate limits
• Enable HTTPS/TLS with valid SSL certificates
• Set up proper CORS origins (restrict to your domain only)
• Configure secure session cookie settings

2. Database Setup

• Choose a PostgreSQL hosting provider (Supabase, RDS, Neon, Railway)
• Create a production database
• Configure connection pooling settings
• Enable automated backups (daily minimum)
• Set up read replicas if expecting high traffic

3. Environment Variables

• Create production .env file with all required variables
• Never commit .env files to version control
• Use secret management service for sensitive values
• Test all environment variables in staging first

4. Domain & DNS

• Purchase and configure domain name
• Set up DNS records (A, CNAME, MX if needed)
• Configure SSL certificates (Let's Encrypt or custom)
• Set up CDN if using static asset hosting

5. Monitoring & Logging

• Set up error tracking (Sentry, Rollbar)
• Configure application logging
• Set up uptime monitoring (Pingdom, UptimeRobot)
• Configure alerting for critical failures

6. CI/CD Pipeline

• Set up automated testing on push
• Configure automated deployment on merge to main
• Set up rollback mechanism
• Configure deployment notifications

---

Option 1: Railway Deployment

Railway is the easiest deployment option with built-in PostgreSQL and container support.

Prerequisites

• Railway account (free tier available)
• GitHub account with repository connected to Railway

Step 1: Connect Repository

1. Log in to Railway
2. Click "New Project" → "Deploy from GitHub repo"
3. Select your TrustOS repository
4. Railway will detect the Docker Compose configuration

Step 2: Configure PostgreSQL

1. Railway will automatically create a PostgreSQL service
2. Click on the PostgreSQL service
3. Copy the connection string
4. Add to your backend environment variables as DATABASE_URL

Step 3: Configure Backend Service

1. Click on the backend service
2. Add environment variables:

   DATABASE_URL=postgresql+asyncpg://...
   SYNC_DATABASE_URL=postgresql://...
   SECRET_KEY=<generate with openssl rand -hex 32>
   OPENAI_API_KEY=sk-...
   AI_PROVIDER=openai
   STORAGE_PATH=/app/storage
   

3. Set the root directory to backend
4. Set the start command to uvicorn app.main:app --host 0.0.0.0 --port $PORT

Step 4: Configure Frontend Service

1. Click "New Service" → "GitHub Repo"
2. Select the same repository
3. Set the root directory to frontend
4. Add environment variable:

   NEXT_PUBLIC_API_URL=https://your-backend-url.railway.app
   

5. Set the start command to npm start
6. Set the build command to npm run build

Step 5: Deploy

1. Click "Deploy" on each service
2. Railway will build and deploy your services
3. Wait for the deployment to complete (2-5 minutes)
4. Access your application at the provided Railway URLs

Step 6: Configure Custom Domain (Optional)

1. Click on your frontend service
2. Go to "Settings" → "Networking"
3. Add your custom domain
4. Update DNS records as instructed by Railway
5. Railway will automatically provision SSL certificates

Railway-Specific Considerations

• Storage: Railway provides ephemeral storage, use Railway Volume for persistent storage
• Database: Railway PostgreSQL includes automated backups
• Scaling: Automatic scaling based on usage
• Cost: Free tier available, then $5/month per service

---

Option 2: Render Deployment

Render offers excellent PostgreSQL support and simple deployment.

Prerequisites

• Render account
• GitHub account with repository

Step 1: Deploy PostgreSQL

1. Log in to Render
2. Click "New" → "PostgreSQL"
3. Choose a database name (e.g., trustos-prod)
4. Select a region closest to your users
5. Choose a plan (Free tier available)
6. Click "Create Database"
7. Copy the internal database URL

Step 2: Deploy Backend

1. Click "New" → "Web Service"
2. Connect your GitHub repository
3. Set the following:
   • Name: trustos-backend
   • Root Directory: backend
   • Build Command: pip install -r requirements.txt
   • Start Command: uvicorn app.main:app --host 0.0.0.0 --port $PORT
4. Add environment variables:

   DATABASE_URL=<your-postgres-url>
   SYNC_DATABASE_URL=<your-postgres-url-sync>
   SECRET_KEY=<generate with openssl rand -hex 32>
   OPENAI_API_KEY=sk-...
   AI_PROVIDER=openai
   STORAGE_PATH=/opt/render/project/storage
   

5. Click "Create Web Service"

Step 3: Deploy Frontend

1. Click "New" → "Web Service"
2. Connect your GitHub repository
3. Set the following:
   • Name: trustos-frontend
   • Root Directory: frontend
   • Build Command: npm run build
   • Start Command: npm start
4. Add environment variable:

   NEXT_PUBLIC_API_URL=https://trustos-backend.onrender.com
   

5. Click "Create Web Service"

Step 4: Initialize Database

1. SSH into your backend service (Render provides this)
2. Run the seed script:

   python seed.py
   

Step 5: Configure Custom Domain

1. Go to your frontend service settings
2. Add your custom domain
3. Update DNS records as instructed
4. Render will automatically provision SSL

Render-Specific Considerations

• Free Tier: Available but spins down after inactivity
• Database: Automated daily backups included
• Storage: Use Render Disk for persistent storage
• Cost: Free tier available, then ~$7/month per service

---

Option 3: VPS Deployment

For full control and cost-effectiveness, deploy to a VPS (DigitalOcean, Linode, AWS EC2, etc.).

Prerequisites

• VPS with Ubuntu 22.04+ (minimum 2GB RAM, 2 CPU)
• Domain name
• SSH access to VPS
• Basic Linux command-line knowledge

Step 1: Prepare VPS

# SSH into your VPS
ssh root@your-vps-ip

# Update system
apt update && apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

# Install Docker Compose
curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

# Install Nginx
apt install nginx -y

# Install Certbot for SSL
apt install certbot python3-certbot-nginx -y

Step 2: Clone Repository

# Install Git
apt install git -y

# Clone repository
cd /opt
git clone https://gitea.thetempleofdoom.com/drjones/trustos.git
cd trustos

Step 3: Configure Environment

# Copy environment template
cp backend/.env.example backend/.env

# Edit with production values
nano backend/.env

Set the following:

DATABASE_URL=postgresql+asyncpg://trustos:strong-password@postgres:5432/trustos
SYNC_DATABASE_URL=postgresql://trustos:strong-password@postgres:5432/trustos
SECRET_KEY=<generate with openssl rand -hex 32>
OPENAI_API_KEY=sk-...
AI_PROVIDER=openai
STORAGE_PATH=/app/storage

Step 4: Configure Docker Compose

Create production docker-compose.yml:

version: '3.8'

services:
  postgres:
    image: postgres:16
    container_name: trustos-postgres
    environment:
      POSTGRES_USER: trustos
      POSTGRES_PASSWORD: strong-password
      POSTGRES_DB: trustos
    volumes:
      - postgres_data:/var/lib/postgresql/data
    restart: unless-stopped
    networks:
      - trustos-network

  backend:
    build:
      context: .
      dockerfile: infra/Dockerfile.backend
    container_name: trustos-backend
    environment:
      DATABASE_URL: postgresql+asyncpg://trustos:strong-password@postgres:5432/trustos
      SYNC_DATABASE_URL: postgresql://trustos:strong-password@postgres:5432/trustos
      SECRET_KEY: ${SECRET_KEY}
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      AI_PROVIDER: openai
      STORAGE_PATH: /app/storage
    depends_on:
      - postgres
    restart: unless-stopped
    networks:
      - trustos-network

  frontend:
    build:
      context: .
      dockerfile: infra/Dockerfile.frontend
    container_name: trustos-frontend
    environment:
      NEXT_PUBLIC_API_URL: https://api.yourdomain.com
    depends_on:
      - backend
    restart: unless-stopped
    networks:
      - trustos-network

volumes:
  postgres_data:

networks:
  trustos-network:
    driver: bridge

Step 5: Start Services

# Start all services
docker-compose up -d

# Check status
docker-compose ps

# View logs
docker-compose logs -f

Step 6: Initialize Database

# Run seed script
docker-compose exec backend python seed.py

Step 7: Configure Nginx

Create Nginx configuration:

nano /etc/nginx/sites-available/trustos

Add the following:

# Frontend (Next.js)
server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

# Backend API
server {
    listen 80;
    server_name api.yourdomain.com;

    location / {
        proxy_pass http://localhost:8000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Enable the site:

ln -s /etc/nginx/sites-available/trustos /etc/nginx/sites-enabled/
nginx -t
systemctl restart nginx

Step 8: Configure SSL with Let's Encrypt

# Obtain SSL certificate
certbot --nginx -d yourdomain.com -d www.yourdomain.com -d api.yourdomain.com

# Certbot will automatically configure Nginx with SSL

# Test auto-renewal
certbot renew --dry-run

Step 9: Set Up Firewall

# Configure UFW
ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable

Step 10: Set Up Monitoring

Install monitoring tools:

# Install htop for resource monitoring
apt install htop -y

# Install fail2ban for security
apt install fail2ban -y

# Configure fail2ban
nano /etc/fail2ban/jail.local

VPS-Specific Considerations

• Backups: Set up automated PostgreSQL backups to S3 or similar
• Updates: Regularly update system and Docker images
• Security: Use SSH keys, disable password authentication
• Cost: ~$20-50/month depending on VPS size

---

Option 4: Kubernetes Deployment

For enterprise-scale deployments with high availability requirements.

Prerequisites

• Kubernetes cluster (AWS EKS, GKE, AKS, or self-hosted)
• kubectl configured
• Helm installed
• Ingress controller (NGINX, Traefik, etc.)

Step 1: Create Namespace

kubectl create namespace trustos

Step 2: Create Secrets

# Create database secret
kubectl create secret generic trustos-db-secret \
  --from-literal=database-url="postgresql+asyncpg://..." \
  --namespace=trustos

# Create API secret
kubectl create secret generic trustos-api-secret \
  --from-literal=secret-key="..." \
  --from-literal=openai-api-key="sk-..." \
  --namespace=trustos

Step 3: Deploy PostgreSQL

Using Helm:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install trustos-postgres bitnami/postgresql \
  --namespace trustos \
  --set auth.password=strong-password \
  --set auth.database=trustos \
  --set persistence.enabled=true

Step 4: Create Backend Deployment

# backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: trustos-backend
  namespace: trustos
spec:
  replicas: 3
  selector:
    matchLabels:
      app: trustos-backend
  template:
    metadata:
      labels:
        app: trustos-backend
    spec:
      containers:
      - name: backend
        image: your-registry/trustos-backend:latest
        ports:
        - containerPort: 8000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: trustos-db-secret
              key: database-url
        - name: SECRET_KEY
          valueFrom:
            secretKeyRef:
              name: trustos-api-secret
              key: secret-key
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: trustos-backend
  namespace: trustos
spec:
  selector:
    app: trustos-backend
  ports:
  - port: 8000
    targetPort: 8000
  type: ClusterIP

Step 5: Create Frontend Deployment

# frontend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: trustos-frontend
  namespace: trustos
spec:
  replicas: 2
  selector:
    matchLabels:
      app: trustos-frontend
  template:
    metadata:
      labels:
        app: trustos-frontend
    spec:
      containers:
      - name: frontend
        image: your-registry/trustos-frontend:latest
        ports:
        - containerPort: 3000
        env:
        - name: NEXT_PUBLIC_API_URL
          value: "https://api.yourdomain.com"
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: trustos-frontend
  namespace: trustos
spec:
  selector:
    app: trustos-frontend
  ports:
  - port: 3000
    targetPort: 3000
  type: ClusterIP

Step 6: Create Ingress

# ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: trustos-ingress
  namespace: trustos
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - yourdomain.com
    - api.yourdomain.com
    secretName: trustos-tls
  rules:
  - host: yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: trustos-frontend
            port:
              number: 3000
  - host: api.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: trustos-backend
            port:
              number: 8000

Step 7: Deploy

kubectl apply -f backend-deployment.yaml
kubectl apply -f frontend-deployment.yaml
kubectl apply -f ingress.yaml

Step 8: Initialize Database

kubectl exec -it deployment/trustos-backend -n trustos -- python seed.py

Kubernetes-Specific Considerations

• High Availability: Multiple replicas across availability zones
• Auto-scaling: Configure Horizontal Pod Autoscaler
• Secrets Management: Use external secret manager (Vault, AWS Secrets Manager)
• Monitoring: Install Prometheus and Grafana
• Logging: Use centralized logging (ELK stack, Loki)
• Cost: Higher due to infrastructure complexity

---

Post-Deployment Steps

1. Verify Deployment

# Check health endpoint
curl https://api.yourdomain.com/health

# Check frontend
curl https://yourdomain.com

# Check API documentation
open https://api.yourdomain.com/docs

2. Run Database Migrations

# If using Alembic
alembic upgrade head

3. Seed Initial Data

# Run seed script
python seed.py

4. Configure Monitoring

Set up monitoring for:

• Application health (uptime, response times)
• Database performance (query times, connection pool)
• Resource usage (CPU, memory, disk)
• Error rates (500 errors, exceptions)

5. Set Up Alerts

Configure alerts for:

• Application downtime
• High error rates
• Database connection issues
• Disk space usage > 80%
• Memory usage > 90%

6. Test User Flows

Test critical user journeys:

• User login
• Dashboard loading
• Finding creation
• Report generation
• AI translation

---

Monitoring & Maintenance

Application Monitoring

Recommended Tools

• Sentry: Error tracking and performance monitoring
• Datadog: Full-stack monitoring
• New Relic: Application performance monitoring
• Prometheus + Grafana: Open-source monitoring stack

Key Metrics to Monitor

• Request rate and response time
• Error rate (4xx, 5xx)
• Database query performance
• AI API usage and costs
• Active users and sessions

Log Management

Recommended Tools

• Loggly: Cloud-based log management
• Papertrail: Simple log aggregation
• ELK Stack: Elasticsearch, Logstash, Kibana
• Loki: Grafana's log aggregation system

Log Levels

• ERROR: Critical errors requiring immediate attention
• WARNING: Issues that should be investigated
• INFO: Normal operational information
• DEBUG: Detailed debugging information (development only)

Regular Maintenance Tasks

Daily

• Review error logs
• Check system resource usage
• Verify backup completion

Weekly

• Review security logs
• Check for dependency updates
• Review performance metrics

Monthly

• Apply security patches
• Review and optimize database queries
• Review AI API costs
• Test disaster recovery procedures

Quarterly

• Security audit
• Performance review
• Capacity planning
• Disaster recovery testing

---

Backup & Recovery

Database Backups

PostgreSQL Backup Strategy

# Daily backup script
#!/bin/bash
DATE=$(date +%Y%m%d)
BACKUP_DIR="/backups/postgres"
pg_dump -h localhost -U trustos trustos | gzip > $BACKUP_DIR/trustos_$DATE.sql.gz

# Keep last 30 days
find $BACKUP_DIR -name "trustos_*.sql.gz" -mtime +30 -delete

Backup Storage

• Store backups in multiple locations (local + cloud)
• Use S3, Glacier, or similar for cloud storage
• Encrypt backups at rest
• Test restore procedures regularly

Application Backups

What to Back Up

• Database dumps
• File storage (PDF reports, uploads)
• Configuration files
• Environment variables (secure storage)

Backup Schedule

• Database: Daily, retain 30 days
• Files: Weekly, retain 90 days
• Config: On change, version control

Recovery Procedures

Database Recovery

# Restore from backup
gunzip < trustos_20240101.sql.gz | psql -h localhost -U trustos trustos

Disaster Recovery

1. Identify the scope of the disaster
2. Determine the point of recovery
3. Restore from the most recent good backup
4. Verify data integrity
5. Test application functionality
6. Monitor for issues

---

Troubleshooting

Common Issues

Application Won't Start

Symptoms: Container exits immediately, 500 errors

Solutions:

• Check logs: docker-compose logs backend
• Verify environment variables
• Check database connectivity
• Verify port availability

Database Connection Errors

Symptoms: "could not connect to server"

Solutions:

• Verify database is running
• Check connection string
• Verify network connectivity
• Check firewall rules

SSL Certificate Issues

Symptoms: Browser warnings, certificate errors

Solutions:

• Verify DNS records are correct
• Check certificate expiration
• Renew with Certbot: certbot renew
• Verify Nginx configuration

High Memory Usage

Symptoms: OOM errors, slow performance

Solutions:

• Check for memory leaks
• Increase container memory limits
• Optimize database queries
• Enable connection pooling

AI Features Not Working

Symptoms: Empty AI translations, errors

Solutions:

• Verify API key is valid
• Check API quota/credits
• Review AI service logs
• Test API key manually

Getting Help

1. Check logs: docker-compose logs -f
2. Review documentation: /docs
3. Check status page: https://status.trustos.com
4. Contact support: support@trustos.com

---

Security Best Practices

Network Security

• Use HTTPS everywhere
• Configure firewall rules
• Use VPN for admin access
• Implement rate limiting
• Use DDoS protection

Application Security

• Keep dependencies updated
• Use strong secrets
• Enable security headers
• Implement CORS properly
• Regular security audits

Data Security

• Encrypt data at rest
• Encrypt data in transit
• Implement access controls
• Regular security training
• Incident response plan

---

Cost Optimization

Railway

• Use free tier for development
• Scale down during off-hours
• Monitor usage regularly
• Use reserved instances for production

Render

• Use free tier for staging
• Optimize build times
• Use appropriate instance sizes
• Monitor resource usage

VPS

• Right-size your instance
• Use spot instances for non-critical workloads
• Implement auto-scaling
• Monitor and optimize resource usage

Kubernetes

• Use resource limits
• Implement auto-scaling
• Use spot instances
• Optimize pod requests/limits

---

Conclusion

This deployment guide covers the most common deployment scenarios for TrustOS. Choose the option that best fits your team's expertise, budget, and scalability requirements.

For additional support or questions, refer to the main README.md or contact the TrustOS development team.