# PM2 Management Guide

Complete guide for managing the Redmine-Gitea Sync Server with PM2.

## Quick Start

### Install PM2
```bash
npm install -g pm2
```

### Start Application
```bash
pm2 start ecosystem.config.cjs
pm2 save
pm2 startup
```

## Essential Commands

### Process Management

```bash
# Start the application
pm2 start ecosystem.config.cjs

# Start in development mode
pm2 start ecosystem.config.cjs --env development

# Stop the application
pm2 stop redmine-gitea-sync

# Restart the application
pm2 restart redmine-gitea-sync

# Reload with zero downtime (cluster mode)
pm2 reload redmine-gitea-sync

# Delete from PM2
pm2 delete redmine-gitea-sync

# Update environment variables and restart
pm2 restart redmine-gitea-sync --update-env
```

### Monitoring

```bash
# List all processes
pm2 list
pm2 ls

# Show detailed information
pm2 show redmine-gitea-sync

# Monitor in real-time
pm2 monit

# View resource usage
pm2 status
```

### Log Management

```bash
# View logs in real-time
pm2 logs redmine-gitea-sync

# View last 100 lines
pm2 logs redmine-gitea-sync --lines 100

# View only errors
pm2 logs redmine-gitea-sync --err

# View only output
pm2 logs redmine-gitea-sync --out

# Clear logs
pm2 flush

# Follow logs with grep
pm2 logs redmine-gitea-sync | grep ERROR
```

### Save and Restore

```bash
# Save current process list
pm2 save

# Resurrect saved process list
pm2 resurrect

# Clear saved process list
pm2 cleardump
```

## Configuration

### Ecosystem File

The `ecosystem.config.cjs` file contains all configuration:

```javascript
module.exports = {
  apps: [{
    name: 'redmine-gitea-sync',
    script: './server.mjs',
    env: {
      // Your environment variables
    },
    instances: 1,
    exec_mode: 'fork',
    max_memory_restart: '300M',
    // ... other settings
  }]
};
```

### Update Configuration

1. Edit `ecosystem.config.cjs`
2. Restart with updated environment:
   ```bash
   pm2 restart redmine-gitea-sync --update-env
   ```

### Multiple Environments

```javascript
// ecosystem.config.cjs
module.exports = {
  apps: [{
    name: 'redmine-gitea-sync',
    script: './server.mjs',
    env: {
      NODE_ENV: 'production',
      PORT: 3002,
    },
    env_development: {
      NODE_ENV: 'development',
      PORT: 3002,
      LOG_LEVEL: 'debug',
    },
    env_staging: {
      NODE_ENV: 'staging',
      PORT: 3003,
    }
  }]
};
```

Start with specific environment:
```bash
pm2 start ecosystem.config.cjs --env development
pm2 start ecosystem.config.cjs --env staging
```

## Auto-Start on Boot

### Setup Startup Script

```bash
# Generate startup script for your system
pm2 startup

# This will output a command like:
# sudo env PATH=$PATH:/usr/bin pm2 startup systemd -u yourusername --hp /home/yourusername

# Run the command it outputs
sudo env PATH=$PATH:/usr/bin pm2 startup systemd -u yourusername --hp /home/yourusername

# Save your process list
pm2 save
```

### Disable Startup

```bash
pm2 unstartup
```

## Clustering

For high-traffic scenarios, use cluster mode:

### Update ecosystem.config.cjs

```javascript
module.exports = {
  apps: [{
    name: 'redmine-gitea-sync',
    script: './server.mjs',
    instances: 'max',  // or specific number like 4
    exec_mode: 'cluster',
    // ... other settings
  }]
};
```

### Commands

```bash
# Start in cluster mode
pm2 start ecosystem.config.cjs

# Scale up/down
pm2 scale redmine-gitea-sync +2  # Add 2 instances
pm2 scale redmine-gitea-sync 4   # Set to 4 instances

# Reload with zero downtime
pm2 reload redmine-gitea-sync
```

## Log Rotation

### Install PM2 Log Rotate Module

```bash
pm2 install pm2-logrotate
```

### Configure Log Rotation

```bash
# Set max file size (default: 10MB)
pm2 set pm2-logrotate:max_size 10M

# Set number of files to keep (default: 10)
pm2 set pm2-logrotate:retain 7

# Enable compression
pm2 set pm2-logrotate:compress true

# Set rotation interval (daily/hourly)
pm2 set pm2-logrotate:rotateInterval '0 0 * * *'  # Daily at midnight

# View configuration
pm2 conf pm2-logrotate
```

## Monitoring and Metrics

### PM2 Plus (Optional)

PM2 Plus provides advanced monitoring:

```bash
# Link to PM2 Plus
pm2 link <secret_key> <public_key>

# Unlink
pm2 unlink
```

### Custom Metrics

Add to your application:

```javascript
const pmx = require('pmx');

// Custom metric
const metric = pmx.probe().metric({
  name: 'Sync Operations',
  value: 0
});

// Increment
metric.set(metric.val() + 1);
```

### System Monitoring

```bash
# Install system monitoring module
pm2 install pm2-server-monit

# View dashboard
# Access via http://localhost:9615
```

## Troubleshooting

### Process Keeps Restarting

```bash
# Check error logs
pm2 logs redmine-gitea-sync --err

# View restart count
pm2 list

# Increase restart delay in ecosystem.config.cjs
restart_delay: 5000,
max_restarts: 5,
```

### High Memory Usage

```bash
# Check memory
pm2 list
pm2 monit

# View detailed info
pm2 show redmine-gitea-sync

# Adjust memory limit in ecosystem.config.cjs
max_memory_restart: '500M',
```

### Application Not Starting

```bash
# Check if port is in use
sudo lsof -i :3002

# Check logs for errors
pm2 logs redmine-gitea-sync --err --lines 50

# Try starting manually to see errors
node server.js
```

### PM2 Not Starting on Boot

```bash
# Verify startup is configured
pm2 startup

# Check systemd status
systemctl status pm2-yourusername

# Regenerate startup script
pm2 unstartup
pm2 startup
pm2 save
```

### Logs Not Appearing

```bash
# Check log file locations
pm2 show redmine-gitea-sync

# Verify log directory exists and is writable
ls -la ./logs/

# Create logs directory if missing
mkdir -p ./logs

# Restart application
pm2 restart redmine-gitea-sync
```

## Advanced Usage

### Watch Mode

Monitor file changes and auto-restart:

```javascript
// ecosystem.config.cjs
watch: true,
watch_delay: 1000,
ignore_watch: ['node_modules', 'logs', '.git'],
```

Or from command line:
```bash
pm2 start ecosystem.config.cjs --watch
```

### Cron Restart

Restart at specific intervals:

```javascript
// ecosystem.config.cjs
cron_restart: '0 3 * * *',  // Every day at 3 AM
```

### Max Restarts

Limit restart attempts:

```javascript
// ecosystem.config.cjs
max_restarts: 10,
min_uptime: '10s',
```

### Environment-Specific Commands

```bash
# Production
pm2 start ecosystem.config.cjs --env production

# Development
pm2 start ecosystem.config.cjs --env development

# Staging
pm2 start ecosystem.config.cjs --env staging
```

## PM2 Runtime Configuration

### Update PM2

```bash
# Update PM2 to latest version
npm install -g pm2@latest

# Update in-memory PM2
pm2 update
```

### PM2 Home Directory

By default: `~/.pm2`

Change location:
```bash
export PM2_HOME=/path/to/custom/location
```

### Reset PM2

```bash
# Kill PM2 daemon
pm2 kill

# Clear everything
rm -rf ~/.pm2

# Restart PM2
pm2 start ecosystem.config.cjs
```

## Integration with CI/CD

### Deploy Script Example

```bash
#!/bin/bash

# Pull latest code
git pull origin main

# Install dependencies
npm install --production

# Restart with zero downtime
pm2 reload ecosystem.config.cjs

# Check if running
pm2 list
```

### GitHub Actions Example

```yaml
name: Deploy

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Deploy to server
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.SSH_KEY }}
          script: |
            cd /opt/redmine-gitea-sync
            git pull
            npm install --production
            pm2 reload ecosystem.config.cjs
```

## Performance Tuning

### Node.js Options

```javascript
// ecosystem.config.cjs
node_args: '--max-old-space-size=2048',
```

### Increase File Descriptors

```bash
# Check current limit
ulimit -n

# Increase limit (temporary)
ulimit -n 10000

# Permanent: Edit /etc/security/limits.conf
* soft nofile 10000
* hard nofile 10000
```

### Process Priority

```bash
# Start with higher priority (lower nice value)
pm2 start ecosystem.config.cjs --node-args="--nice=-10"
```

## Backup and Restore

### Backup PM2 Configuration

```bash
# Backup process list
pm2 save

# Backup ecosystem file
cp ecosystem.config.cjs ecosystem.config.cjs.backup

# Backup PM2 directory
tar -czf pm2-backup.tar.gz ~/.pm2
```

### Restore

```bash
# Restore PM2 directory
tar -xzf pm2-backup.tar.gz -C ~/

# Resurrect processes
pm2 resurrect
```

## Security Best Practices

### Run as Dedicated User

```bash
# Create user
sudo useradd -r -s /bin/false pm2user

# Set ownership
sudo chown -R pm2user:pm2user /opt/redmine-gitea-sync

# Run PM2 as user
sudo su - pm2user -s /bin/bash -c "pm2 start /opt/redmine-gitea-sync/ecosystem.config.cjs"
sudo su - pm2user -s /bin/bash -c "pm2 save"

# Setup startup as that user
sudo env PATH=$PATH:/usr/bin pm2 startup systemd -u pm2user --hp /home/pm2user
```

### Secure Environment Variables

Never commit sensitive data to version control:

```bash
# Protect ecosystem file
chmod 600 ecosystem.config.cjs

# Use external .env file
# In ecosystem.config.cjs:
require('dotenv').config();

env: {
  REDMINE_API_KEY: process.env.REDMINE_API_KEY,
  GITEA_TOKEN: process.env.GITEA_TOKEN,
}
```

## Quick Reference

### Most Common Commands

```bash
# Start
pm2 start ecosystem.config.cjs

# Status
pm2 list

# Logs
pm2 logs

# Restart
pm2 restart redmine-gitea-sync

# Stop
pm2 stop redmine-gitea-sync

# Monitor
pm2 monit

# Save
pm2 save
```

### Useful Aliases

Add to `~/.bashrc`:

```bash
alias pm2-status='pm2 list'
alias pm2-logs='pm2 logs redmine-gitea-sync'
alias pm2-restart='pm2 restart redmine-gitea-sync'
alias pm2-sync='cd /opt/redmine-gitea-sync && pm2 logs redmine-gitea-sync'
```

## Additional Resources

- [PM2 Official Documentation](https://pm2.keymetrics.io/)
- [PM2 GitHub Repository](https://github.com/Unitech/pm2)
- [PM2 Plus (Monitoring)](https://pm2.io/)
- [PM2 Cheat Sheet](https://pm2.keymetrics.io/docs/usage/quick-start/)

## Support

For PM2-specific issues:
- Check PM2 logs: `pm2 logs`
- Consult PM2 documentation
- Search PM2 GitHub issues

For application issues:
- Check application logs: `pm2 logs redmine-gitea-sync`
- Review main documentation
- Check server health: `curl http://localhost:3002/health`