Maintenance & Operations
Routine tasks and troubleshooting for TimeTiles administrators.
Daily Operations
Health Monitoring
Check system health regularly:
# API health check
curl https://your-domain.com/api/health
# Container status (production)
docker-compose -f deployment/docker-compose.prod.yml ps
# Database connectivity
docker-compose -f deployment/docker-compose.prod.yml exec postgres pg_isready
# View recent logs
./deploy.sh logs --tail=100Backup Procedures
Quick Commands
# Database backup (default)
./deploy.sh backup
# Full backup (database + uploads)
./deploy.sh backup full
# Setup automatic daily backups at 2 AM
./deploy.sh backup auto
# List available backups
./deploy.sh backup list
# Clean old backups (>30 days)
./deploy.sh backup cleanRestore from Backup
# List available backups
./deploy.sh restore
# Restore database
./deploy.sh restore db-20241220-143022.sql.gz
# Restore database and uploads
./deploy.sh restore db-20241220-143022.sql.gz uploads-20241220-143022.tar.gzThe backup system automatically:
- Keeps only the last 30 database backups
- Stores backups in
deployment/backups/ - Names files with timestamps for easy identification
- Compresses backups to save space
Updates & Upgrades
Application Updates
- Backup first - Always create a backup before updating
- Pull latest code - Get the latest approved changes
- Rebuild and deploy - Use deployment commands to update
- Run migrations - Apply any database schema changes
- Verify health - Check that the application is running correctly
See Deployment Commands for detailed command reference.
Update Workflow
# 1. Backup first
./deploy.sh backup
# 2. Pull latest code
git pull origin main
# 3. Use update command (handles rebuild, migrations, restart)
./deploy.sh update
# 4. Verify health
curl https://your-domain.com/api/healthNote: Dependency updates should be handled by developers through pull requests, not directly in production.
Database Migrations
Database migrations run automatically on container startup in production via Payload’s prodMigrations feature.
# Check migration status in health endpoint
curl https://your-domain.com/api/health | jq '.migrations'
# View migration logs
./deploy.sh logs | grep -i migratNote: Migrations are applied automatically when the web container starts, so no manual intervention is needed.
Performance Optimization
Database Maintenance
Run regularly (weekly recommended):
# Connect to database
docker-compose -f deployment/docker-compose.prod.yml exec postgres psql -U timetiles_user timetiles
# Vacuum and analyze
VACUUM ANALYZE;
# Reindex for performance
REINDEX DATABASE timetiles;
# Update table statistics
ANALYZE;Query Performance
-- Enable query stats
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- Find slow queries
SELECT
mean_exec_time,
calls,
query
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 10;
-- Check index usage
SELECT
schemaname,
tablename,
indexname,
idx_scan
FROM pg_stat_user_indexes
ORDER BY idx_scan;Monitoring
Health Checks
Monitor application health regularly:
# Check overall health
curl https://your-domain.com/api/health
# Check service status
./deploy.sh status
# Verify services are running
docker-compose -f deployment/docker-compose.prod.yml psRecommended monitoring schedule:
- Health endpoint: Every 5 minutes
- Service status: Every hour
- Log review: Daily
- Resource usage: Weekly
Log Management
# View live logs
./deploy.sh logs -f
# Check specific service
docker-compose -f deployment/docker-compose.prod.yml logs web --tail=100
docker-compose -f deployment/docker-compose.prod.yml logs postgres --tail=50
# Search logs
docker-compose -f deployment/docker-compose.prod.yml logs | grep ERROR
docker-compose -f deployment/docker-compose.prod.yml logs | grep "import.*failed"Disk Usage
# Check disk space
df -h
# Find large files
du -h /var/lib/docker/volumes | sort -rh | head -20
# Clean up Docker
docker system prune -a --volumesResource Usage
# Container stats
docker stats
# Database size
docker-compose -f deployment/docker-compose.prod.yml exec postgres \
psql -U timetiles_user -d timetiles -c \
"SELECT pg_size_pretty(pg_database_size('timetiles'));"
# Table sizes
docker-compose -f deployment/docker-compose.prod.yml exec postgres \
psql -U timetiles_user -d timetiles -c \
"SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size
FROM pg_tables
WHERE schemaname = 'payload'
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC;"Troubleshooting
Common Issues
Application Won’t Start
# Check logs
./deploy.sh logs web
# Verify environment
docker-compose -f deployment/docker-compose.prod.yml exec web env | grep -E "DATABASE|PAYLOAD"
# Test database connection
docker-compose -f deployment/docker-compose.prod.yml exec web \
psql $DATABASE_URL -c "SELECT 1"Import Failures
# Check import job status
docker-compose -f deployment/docker-compose.prod.yml exec web \
psql $DATABASE_URL -c "SELECT * FROM payload.import_jobs ORDER BY created_at DESC LIMIT 5;"
# View error details
docker-compose -f deployment/docker-compose.prod.yml logs web | grep -A5 "import.*error"
# Reset stuck imports
docker-compose -f deployment/docker-compose.prod.yml exec web \
psql $DATABASE_URL -c "UPDATE payload.import_jobs SET status = 'failed' WHERE status = 'processing' AND updated_at < NOW() - INTERVAL '1 hour';"High Memory Usage
# Check Node.js memory
docker-compose -f deployment/docker-compose.prod.yml exec web \
node -e "console.log(process.memoryUsage())"
# Restart to clear memory
docker-compose -f deployment/docker-compose.prod.yml restart web
# Increase memory limit if needed
# Edit deployment/.env.production
NODE_OPTIONS="--max-old-space-size=4096"SSL Certificate Issues
# Check certificate expiry
docker-compose -f deployment/docker-compose.prod.yml exec nginx \
openssl x509 -in /etc/letsencrypt/live/your-domain.com/cert.pem -text -noout | grep "Not After"
# Force renewal
docker-compose -f deployment/docker-compose.prod.yml exec certbot \
certbot renew --force-renewal
# Restart nginx
docker-compose -f deployment/docker-compose.prod.yml restart nginxDebug Mode
Enable detailed logging for troubleshooting:
# Edit deployment/.env.production
LOG_LEVEL=debug
# Restart application
./deploy.sh restart
# View debug logs
./deploy.sh logs web | grep DEBUGSecurity Updates
SSL Certificate Renewal
Certificates auto-renew, but verify monthly:
# Check renewal
docker-compose -f deployment/docker-compose.prod.yml exec certbot \
certbot certificates
# Test renewal
docker-compose -f deployment/docker-compose.prod.yml exec certbot \
certbot renew --dry-runSecurity Patches
# Update base images
docker-compose -f deployment/docker-compose.prod.yml pull
./deploy.sh down
./deploy.sh up
# Check for vulnerabilities
pnpm auditGetting Help
- Documentation: Review this guide and deployment docs
- Issues: GitHub Issues
- Community: GitHub Discussions
- Logs: Always check logs first with
./deploy.sh logs
Last updated on