Upgrades

Contract Lucidity is designed for straightforward upgrades. The backend automatically runs database migrations on startup, and container rebuilds capture all code and dependency changes. This guide covers the standard upgrade procedure, zero-downtime strategies, and rollback considerations.

How Updates Work

The upgrade sequence:

1. Pull latest code -- git pull or download the release archive
2. Rebuild containers -- docker compose build compiles new images with updated code and dependencies
3. Restart services -- docker compose up -d replaces running containers with the new images
4. Automatic migration -- The backend's lifespan handler runs alembic upgrade head on startup, applying any pending database schema changes
5. Seed data -- Idempotent seed scripts run to ensure default data (admin user, system config) exists

Standard Upgrade Procedure

Pre-Upgrade Checklist

Before upgrading, verify:

• You have a recent database backup (see Backup & Disaster Recovery)
• No documents are actively being processed (check queue depth)
• You know the current version / commit hash for rollback reference

# Check for in-flight documents
docker exec cl-redis redis-cli LLEN celery
# Should return 0

# Check for documents in non-terminal states
docker exec cl-postgres psql -U cl_user -d contract_lucidity \
  -c "SELECT pipeline_status, count(*) FROM documents
      WHERE pipeline_status NOT IN ('complete', 'failed')
      GROUP BY pipeline_status;"
# Should return no rows

# Record current commit for rollback reference
cd /path/to/contract-lucidity
git rev-parse HEAD

Step-by-Step Upgrade

# 1. Pull the latest code
cd /path/to/contract-lucidity
git pull origin main

# 2. Review what changed (optional but recommended)
git log --oneline HEAD~10..HEAD

# 3. Rebuild all containers
docker compose build

# 4. Restart services (backend will auto-migrate)
docker compose up -d

# 5. Watch the backend logs for migration success
docker logs -f cl-backend 2>&1 | head -50
# Look for: "INFO  [alembic.runtime.migration] Running upgrade ... -> ..."
# And: "Application startup complete."

# 6. Verify health
curl -s https://contractlucidity.com/api/health
# Expected: {"status": "healthy", "service": "Contract Lucidity"}

# 7. Verify the frontend loads correctly
# Open https://contractlucidity.com in a browser

tip

If only backend code changed (no frontend changes), you can selectively rebuild:

docker compose build cl-backend cl-worker
docker compose up -d cl-backend cl-worker

This is faster and avoids any frontend downtime.

Zero-Downtime Upgrade

For production deployments where uptime is critical, use a rolling upgrade strategy.

Single-Server Rolling Update

# 1. Pull code and rebuild images (does not affect running containers)
git pull origin main
docker compose build

# 2. Restart the worker first (drain existing tasks)
docker compose stop cl-worker
# Wait for any in-flight tasks to complete (check logs)
docker compose up -d cl-worker

# 3. Restart the backend (brief interruption -- < 30 seconds)
docker compose up -d cl-backend
# Migrations run during startup; traffic is interrupted until complete

# 4. Restart the frontend last
docker compose up -d cl-frontend

Migration Downtime

If the upgrade includes database migrations, the backend will be unavailable for the duration of the migration run. Simple migrations (adding columns, indices) typically complete in under 5 seconds. Data migrations on large tables may take longer.

Multi-Instance Rolling Update

If you run multiple backend/frontend instances behind a load balancer:

1. Build new images on all hosts
2. Run migrations from one backend instance: docker exec cl-backend-1 alembic upgrade head
3. Upgrade backend instances one at a time, letting the load balancer drain connections
4. Upgrade worker instances one at a time
5. Upgrade frontend instances one at a time

Database Migration Safety

How Migrations Run

CL uses Alembic for database schema management. Key details:

• Migrations run automatically on every backend startup via alembic upgrade head
• Alembic tracks applied migrations in the alembic_version table
• Migrations are idempotent at the Alembic level -- running upgrade head when already at head is a no-op
• The pgvector extension is ensured before migrations run (CREATE EXTENSION IF NOT EXISTS vector)

Migration Configuration

From backend/alembic/env.py:

• The database URL is sourced from the application's Settings (which reads from .env)
• The synchronous database URL (postgresql+psycopg2://...) is used for migrations
• All application models are imported via from app.models import Base, ensuring Alembic knows the full schema

What If a Migration Fails?

If a migration fails mid-way:

1. The backend will not start (the subprocess.run(..., check=True) call raises an exception)
2. Check the backend logs for the specific migration error:

   docker logs cl-backend 2>&1 | grep -A 5 "alembic"

3. Common migration failures:
   • Column already exists -- Migration was partially applied. Fix by stamping: alembic stamp <revision>
   • Permission denied -- Database user lacks ALTER TABLE privileges
   • Lock timeout -- Long-running queries blocking the migration. Retry after killing blocking queries

danger

Never manually edit the alembic_version table unless you fully understand the migration chain. Incorrect stamping can leave your database in an inconsistent state.

Rolling Back

Rollback is Not Recommended

Database migrations are generally forward-only. While Alembic supports downgrade operations, CL's migrations may not all have downgrade paths implemented. The safest approach is to test upgrades in a staging environment first.

If You Must Roll Back

Option 1: Code-Only Rollback (No Migration Changes)

If the upgrade did not include database migrations, you can safely revert the code:

# Revert to the previous commit
git checkout <previous-commit-hash>

# Rebuild and restart
docker compose build
docker compose up -d

Option 2: Full Rollback (With Migration Changes)

If the upgrade included database migrations:

1. Restore from backup (safest):

   # Stop services
   docker compose stop cl-backend cl-worker
   
   # Restore database from pre-upgrade backup
   docker exec -i cl-postgres pg_restore \
     -U cl_user -d contract_lucidity --clean \
     < /opt/backups/cl-postgres/pre_upgrade_backup.dump
   
   # Revert code
   git checkout <previous-commit-hash>
   
   # Rebuild and restart
   docker compose build
   docker compose up -d

2. Alembic downgrade (if downgrade paths exist):

   # Check available revisions
   docker exec cl-backend alembic history
   
   # Downgrade to a specific revision
   docker exec cl-backend alembic downgrade <target-revision>
   
   # Revert code and restart
   git checkout <previous-commit-hash>
   docker compose build
   docker compose up -d

Version Compatibility Notes

Python Dependencies

The requirements.txt is baked into the Docker image at build time. When upgrading, docker compose build installs the latest pinned dependencies. If a new version adds or updates dependencies, the rebuild handles this automatically.

PostgreSQL Version

CL uses pgvector/pgvector:pg16 (PostgreSQL 16). Major PostgreSQL version upgrades (e.g., 16 to 17) require a separate data migration process:

# This is a PostgreSQL upgrade, not a CL upgrade
# 1. pg_dump from the old version
# 2. Start a new pg17 container
# 3. pg_restore into the new container
# 4. Update docker-compose.yml to use the new image

Redis Version

CL uses redis:7-alpine. Redis is used only as a message broker and result backend. Upgrading Redis is straightforward -- just update the image tag and restart.

Node.js / Next.js

The frontend image is built with its own Node.js version. No action is required unless the upgrade notes specify a Node.js version change.

Upgrade Notifications

Checking for Updates

# If deploying from git
cd /path/to/contract-lucidity
git fetch origin
git log HEAD..origin/main --oneline
# Shows commits available to pull

Release Notes

Before upgrading, always review the release notes or commit history for:

• Breaking changes -- environment variable renames, API changes
• New environment variables -- add to your .env before upgrading
• Migration warnings -- large data migrations that may take time
• Dependency changes -- new system packages or Python dependencies

Upgrade Checklist Summary

Step	Command	Verify
1. Back up database	pg_dump	Backup file exists and is > 0 bytes
2. Drain queue	Wait for LLEN celery = 0	No in-flight documents
3. Pull code	git pull origin main	No merge conflicts
4. Review changes	git log	No breaking changes
5. Rebuild	docker compose build	Build completes without errors
6. Restart	docker compose up -d	All containers running
7. Check migrations	docker logs cl-backend	"Running upgrade" messages, no errors
8. Health check	curl /api/health	{"status": "healthy"}
9. Smoke test	Upload a test document	Processes to COMPLETE
10. Monitor	Watch logs for 15 minutes	No errors