From c86d27d59e9d7f4ce347f86e6a156540cf109abd Mon Sep 17 00:00:00 2001
From: Thomas Scott <tombomb122@noreply.localhost>
Date: Wed, 17 Dec 2025 11:53:24 +0000
Subject: [PATCH] Add TROUBLESHOOTING.md

Signed-off-by: Thomas Scott <tombomb122@noreply.localhost>
---
 TROUBLESHOOTING.md | 717 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 717 insertions(+)
 create mode 100644 TROUBLESHOOTING.md

diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md
new file mode 100644
index 0000000..cda92c6
--- /dev/null
+++ b/TROUBLESHOOTING.md
@@ -0,0 +1,717 @@
+# Troubleshooting Guide
+
+Common issues and their solutions for the Redmine-Gitea Sync Server.
+
+## Table of Contents
+
+- [422 Errors](#422-errors)
+- [Connection Issues](#connection-issues)
+- [Sync Loop Issues](#sync-loop-issues)
+- [Performance Issues](#performance-issues)
+- [PM2 Issues](#pm2-issues)
+
+---
+
+## Label and Field Mapping Issues
+
+### Issue: Priority/Tracker Not Syncing from Gitea to Redmine
+
+**Symptoms:**
+- Gitea labels like "bug", "High Priority" not mapping to Redmine fields
+- All Redmine issues default to Bug/Normal priority
+
+**Cause:** Gitea uses labels for everything, but Redmine has distinct `tracker_id` and `priority_id` fields.
+
+**Solution:** The server now automatically maps labels back to Redmine fields:
+
+**Tracker Mapping:**
+- `bug` label → Tracker ID 1 (Bug)
+- `feature` label → Tracker ID 2 (Feature)  
+- `support` label → Tracker ID 3 (Support)
+
+**Priority Mapping:**
+- `Low Priority` → Priority ID 1
+- `Normal Priority` → Priority ID 2
+- `High Priority` → Priority ID 3
+- `Urgent` → Priority ID 4
+- `Immediate` → Priority ID 5
+
+**Verification:**
+```bash
+# Create issue in Gitea with labels "feature" and "High Priority"
+# Check Redmine - should show as Feature with High priority
+
+# Update to latest version
+git pull
+pm2 restart redmine-gitea-sync
+```
+
+### Issue: Comments Not Syncing from Redmine to Gitea
+
+**Symptoms:**
+- Add note/journal in Redmine
+- Comment doesn't appear in Gitea
+- Gitea → Redmine comments work fine
+
+**Causes:**
+1. Redmine webhook not configured for journal updates
+2. Journal has no notes (only field changes)
+3. Cache blocking the sync
+
+**Solutions:**
+
+1. **Verify Redmine Webhook Events**
+   - Go to: Administration → Settings → Repositories → Webhooks
+   - Ensure "Issues updated" is checked
+   - Redmine should send journals as part of issue updates
+
+2. **Check Logs**
+   ```bash
+   pm2 logs redmine-gitea-sync
+   
+   # After adding a Redmine note, you should see:
+   # "Processing Redmine issue #X"
+   # "Synced Redmine journal #Y to Gitea comment on issue #Z"
+   ```
+
+3. **Verify Journal Has Content**
+   - Empty notes (only field changes) won't sync
+   - Add actual text to the note/journal
+
+4. **Check Journal Structure**
+   ```bash
+   # Enable debug logging
+   pm2 start ecosystem.config.cjs --env development
+   
+   # Look for journal processing logs
+   pm2 logs redmine-gitea-sync | grep journal
+   ```
+
+5. **Clear Cache**
+   ```bash
+   curl -X POST http://localhost:3002/cache/clear
+   ```
+
+**Verification:**
+```bash
+# 1. Add a note in Redmine with text content
+# 2. Watch logs
+pm2 logs redmine-gitea-sync
+
+# Should see:
+# [INFO] Processing Redmine issue #X
+# [INFO] Updating existing Gitea issue #Y
+# [INFO] Synced Redmine journal #Z to Gitea comment on issue #Y
+
+# 3. Check Gitea issue - comment should appear with:
+# [Redmine Journal #Z]
+# **username** commented:
+# 
+# Your note text
+```
+
+---
+
+## 422 Errors
+
+### Issue: "Request failed with status code 422" (Redmine → Gitea)
+
+**Symptoms:**
+```
+[ERROR] Redmine to Gitea sync failed: Request failed with status code 422
+```
+
+**Cause:** Gitea API validation failure. The request payload contains invalid data.
+
+**Common Causes:**
+
+1. **Invalid Label Format**
+   - Labels must be set separately using label IDs, not in the issue creation payload
+   - Fixed in v2.0.0+
+
+2. **Invalid Date Format**
+   - Gitea expects ISO 8601 format: `2025-12-17T00:00:00Z`
+   - Fixed in v2.0.0+
+
+3. **Invalid Assignee**
+   - Must be a valid Gitea username (string)
+   - User must exist in Gitea
+
+4. **Invalid Milestone**
+   - Must be a valid milestone ID (number)
+   - Milestone must exist in the repository
+
+**Solutions:**
+
+1. **Update to Latest Version**
+   ```bash
+   git pull
+   npm install
+   pm2 restart redmine-gitea-sync
+   ```
+
+2. **Enable Verbose Logging**
+   ```javascript
+   // In ecosystem.config.cjs
+   env: {
+     LOG_LEVEL: 'debug',
+     LOG_VERBOSE: 'true',
+   }
+   ```
+   
+   Restart and check logs:
+   ```bash
+   pm2 restart redmine-gitea-sync --update-env
+   pm2 logs redmine-gitea-sync
+   ```
+   
+   Look for: `Validation error details: {...}`
+
+3. **Disable Problematic Features Temporarily**
+   ```javascript
+   // In ecosystem.config.cjs
+   env: {
+     SYNC_LABELS: 'false',
+     SYNC_MILESTONES: 'false',
+   }
+   ```
+
+4. **Verify Assignee Exists**
+   ```bash
+   # Test if user exists in Gitea
+   curl -H "Authorization: token YOUR_TOKEN" \
+     https://gitea.example.com/api/v1/users/username
+   ```
+
+5. **Check Date Formats**
+   - Ensure Redmine due dates are valid
+   - Server will now auto-convert to ISO format
+
+**Verification:**
+```bash
+# Watch logs in real-time
+pm2 logs redmine-gitea-sync
+
+# Create a test issue in Redmine
+# Should see: "Successfully synced Redmine issue #X to Gitea issue #Y"
+```
+
+### Issue: "Due date is not a valid date" (Gitea → Redmine)
+
+**Symptoms:**
+```
+[ERROR] Gitea to Redmine sync failed: Request failed with status code 422
+[ERROR] Validation error details: {"errors":["Due date is not a valid date"]}
+```
+
+**Cause:** Redmine expects dates in `YYYY-MM-DD` format (no time component), but Gitea sends ISO 8601 format with time.
+
+**Solution:** This is automatically fixed in v2.0.0+. The server now:
+- Extracts only the date portion from ISO timestamps
+- Validates dates before sending to Redmine
+- Handles invalid date formats gracefully with warnings
+
+**Manual Fix:**
+```bash
+git pull
+pm2 restart redmine-gitea-sync
+```
+
+**Verification:**
+```bash
+# Create issue in Gitea with due date
+# Watch logs - should see successful sync
+pm2 logs redmine-gitea-sync
+
+# Check Redmine - due date should appear correctly
+```
+
+---
+
+## Connection Issues
+
+### Issue: Cannot Connect to Redmine
+
+**Symptoms:**
+```
+[ERROR] Redmine connection test failed
+```
+
+**Solutions:**
+
+1. **Verify API URL**
+   ```bash
+   # Test Redmine API
+   curl -H "X-Redmine-API-Key: YOUR_KEY" \
+     https://redmine.example.com/issues.json
+   ```
+
+2. **Check API Key**
+   - Log into Redmine
+   - Go to: My Account → API access key
+   - Verify key matches `REDMINE_API_KEY` in config
+
+3. **Test Network Connectivity**
+   ```bash
+   # From server, test connection
+   telnet redmine.example.com 443
+   ping redmine.example.com
+   ```
+
+4. **Check Firewall**
+   ```bash
+   # Ensure outbound HTTPS is allowed
+   sudo ufw status
+   ```
+
+5. **Verify SSL Certificate**
+   ```bash
+   # Test SSL
+   openssl s_client -connect redmine.example.com:443
+   ```
+
+### Issue: Cannot Connect to Gitea
+
+**Symptoms:**
+```
+[ERROR] Gitea connection test failed
+```
+
+**Solutions:**
+
+1. **Verify API URL**
+   - Should NOT include `/api/v1`
+   - Correct: `https://gitea.example.com`
+   - Incorrect: `https://gitea.example.com/api/v1`
+
+2. **Check Token**
+   ```bash
+   # Test Gitea API
+   curl -H "Authorization: token YOUR_TOKEN" \
+     https://gitea.example.com/api/v1/user
+   ```
+
+3. **Verify Token Permissions**
+   - Token must have: `repo`, `write:issue`
+   - Regenerate token if needed
+
+4. **Check Owner Name**
+   ```bash
+   # Verify owner exists
+   curl -H "Authorization: token YOUR_TOKEN" \
+     https://gitea.example.com/api/v1/users/YOUR_OWNER
+   ```
+
+---
+
+## Sync Loop Issues
+
+### Issue: Infinite Sync Loop
+
+**Symptoms:**
+- Issues constantly updating
+- High CPU usage
+- Logs showing rapid back-and-forth updates
+
+**Cause:** Cache not preventing loops properly.
+
+**Solutions:**
+
+1. **Increase Cache TTL**
+   ```javascript
+   // In ecosystem.config.cjs
+   env: {
+     CACHE_TTL: '60000',  // 60 seconds
+   }
+   ```
+
+2. **Clear Cache**
+   ```bash
+   curl -X POST http://localhost:3002/cache/clear
+   ```
+
+3. **Verify Webhook Configuration**
+   - Ensure webhooks are not triggering on every change
+   - Check webhook logs in Redmine/Gitea
+
+4. **Check for Multiple Instances**
+   ```bash
+   pm2 list
+   # Should show only one instance
+   ```
+
+### Issue: Updates Not Syncing
+
+**Symptoms:**
+- Changes made but not appearing on other platform
+- No errors in logs
+
+**Solutions:**
+
+1. **Check Cache**
+   ```bash
+   # Cache might be blocking legitimate updates
+   curl -X POST http://localhost:3002/cache/clear
+   ```
+
+2. **Verify Webhooks Are Firing**
+   ```bash
+   # Watch logs
+   pm2 logs redmine-gitea-sync
+
+   # Make a change in Redmine
+   # Should see: "Processing Redmine issue #X"
+   ```
+
+3. **Test Webhooks Manually**
+   ```bash
+   # Test Redmine webhook
+   curl -X POST http://localhost:3002/webhook/redmine \
+     -H "Content-Type: application/json" \
+     -d @test-payload.json
+
+   # Test Gitea webhook
+   curl -X POST http://localhost:3002/webhook/gitea \
+     -H "Content-Type: application/json" \
+     -d @gitea-payload.json
+   ```
+
+4. **Check Network Connectivity**
+   - Ensure Redmine/Gitea can reach sync server
+   - Test from Redmine/Gitea servers:
+   ```bash
+   curl http://sync-server:3002/health
+   ```
+
+---
+
+## Performance Issues
+
+### Issue: High Memory Usage
+
+**Symptoms:**
+```bash
+pm2 list
+# Shows high memory usage (>300MB)
+```
+
+**Solutions:**
+
+1. **Increase Memory Limit**
+   ```javascript
+   // In ecosystem.config.cjs
+   max_memory_restart: '500M',
+   ```
+
+2. **Check for Memory Leaks**
+   ```bash
+   pm2 monit
+   # Watch memory over time
+   ```
+
+3. **Reduce Cache Size**
+   ```javascript
+   env: {
+     CACHE_TTL: '15000',  // Reduce to 15 seconds
+   }
+   ```
+
+4. **Restart Regularly**
+   ```javascript
+   // In ecosystem.config.cjs
+   cron_restart: '0 3 * * *',  // Daily at 3 AM
+   ```
+
+### Issue: Slow Sync Performance
+
+**Symptoms:**
+- Sync takes several seconds
+- Timeouts occurring
+
+**Solutions:**
+
+1. **Increase Timeouts**
+   ```javascript
+   env: {
+     REDMINE_TIMEOUT: '60000',
+     GITEA_TIMEOUT: '60000',
+   }
+   ```
+
+2. **Reduce Retry Attempts**
+   ```javascript
+   env: {
+     RETRY_ATTEMPTS: '2',
+     RETRY_DELAY: '500',
+   }
+   ```
+
+3. **Check Network Latency**
+   ```bash
+   # Test latency
+   ping redmine.example.com
+   ping gitea.example.com
+   ```
+
+4. **Disable Heavy Features**
+   ```javascript
+   env: {
+     SYNC_LABELS: 'false',
+     SYNC_MILESTONES: 'false',
+     SYNC_CUSTOM_FIELDS: 'false',
+   }
+   ```
+
+---
+
+## PM2 Issues
+
+### Issue: Process Keeps Restarting
+
+**Symptoms:**
+```bash
+pm2 list
+# Shows high restart count
+```
+
+**Solutions:**
+
+1. **Check Error Logs**
+   ```bash
+   pm2 logs redmine-gitea-sync --err --lines 50
+   ```
+
+2. **Increase Restart Delay**
+   ```javascript
+   // In ecosystem.config.cjs
+   restart_delay: 5000,
+   max_restarts: 5,
+   ```
+
+3. **Verify Configuration**
+   ```bash
+   # Test configuration
+   node server.js
+   # Should start without errors
+   ```
+
+### Issue: PM2 Not Starting on Boot
+
+**Symptoms:**
+- Server boots but sync server not running
+
+**Solutions:**
+
+1. **Regenerate Startup Script**
+   ```bash
+   pm2 unstartup
+   pm2 startup
+   pm2 save
+   ```
+
+2. **Check Systemd Service**
+   ```bash
+   systemctl status pm2-yourusername
+   journalctl -u pm2-yourusername -n 50
+   ```
+
+3. **Verify PM2 Save**
+   ```bash
+   pm2 save
+   ls ~/.pm2/dump.pm2
+   ```
+
+### Issue: Logs Not Appearing
+
+**Symptoms:**
+```bash
+pm2 logs redmine-gitea-sync
+# Shows no output
+```
+
+**Solutions:**
+
+1. **Check Log Directory**
+   ```bash
+   ls -la ./logs/
+   # Should show error.log and out.log
+   ```
+
+2. **Create Logs Directory**
+   ```bash
+   mkdir -p logs
+   chmod 755 logs
+   pm2 restart redmine-gitea-sync
+   ```
+
+3. **Check Permissions**
+   ```bash
+   # Ensure PM2 user can write to logs
+   chown -R $USER:$USER logs/
+   ```
+
+4. **Verify PM2 Config**
+   ```javascript
+   // In ecosystem.config.cjs
+   error_file: './logs/error.log',
+   out_file: './logs/out.log',
+   ```
+
+---
+
+## Webhook Issues
+
+### Issue: Webhook Not Triggering
+
+**Symptoms:**
+- No logs when creating/updating issues
+- Changes not syncing
+
+**Solutions:**
+
+1. **Verify Webhook URL**
+   - Redmine: `http://your-server:3002/webhook/redmine`
+   - Gitea: `http://your-server:3002/webhook/gitea`
+
+2. **Check Webhook Status**
+   - In Gitea: Repository → Settings → Webhooks
+   - Look for delivery history and error messages
+
+3. **Test Webhook Delivery**
+   ```bash
+   # From Redmine/Gitea server
+   curl -X POST http://sync-server:3002/webhook/redmine \
+     -H "Content-Type: application/json" \
+     -d '{"payload":{"issue":{"id":1,"subject":"Test"}}}'
+   ```
+
+4. **Check Firewall**
+   ```bash
+   # Ensure port 3002 is open
+   sudo ufw status
+   sudo ufw allow 3002/tcp
+   ```
+
+5. **Use Reverse Proxy**
+   - Configure Nginx to forward webhooks
+   - Use HTTPS for production
+
+---
+
+## Project Mapping Issues
+
+### Issue: Project Not Found
+
+**Symptoms:**
+```
+[WARN] No Gitea repo mapping found for Redmine project xyz
+[ERROR] Redmine project xyz not found
+```
+
+**Solutions:**
+
+1. **Configure Project Mapping**
+   ```javascript
+   // In ecosystem.config.cjs
+   env: {
+     PROJECT_MAPPING: '{"GiteaRepo":"redmine-project-id"}',
+   }
+   ```
+
+2. **Verify Project Identifiers**
+   ```bash
+   # Check Redmine project identifier
+   curl -H "X-Redmine-API-Key: KEY" \
+     https://redmine.example.com/projects/PROJECT_ID.json
+
+   # Check Gitea repository name
+   curl -H "Authorization: token TOKEN" \
+     https://gitea.example.com/api/v1/repos/OWNER/REPO
+   ```
+
+3. **Use Automatic Mapping**
+   - Leave `PROJECT_MAPPING` empty
+   - Server will auto-convert names
+
+---
+
+## Debug Mode
+
+### Enable Maximum Verbosity
+
+```javascript
+// In ecosystem.config.cjs
+env: {
+  NODE_ENV: 'development',
+  LOG_LEVEL: 'debug',
+  LOG_VERBOSE: 'true',
+}
+```
+
+Restart:
+```bash
+pm2 restart redmine-gitea-sync --update-env
+```
+
+Or use development environment:
+```bash
+pm2 start ecosystem.config.cjs --env development
+```
+
+### View Detailed Logs
+
+```bash
+# Real-time logs with timestamps
+pm2 logs redmine-gitea-sync --timestamp
+
+# Last 200 lines
+pm2 logs redmine-gitea-sync --lines 200
+
+# Only errors
+pm2 logs redmine-gitea-sync --err
+
+# Follow and filter
+pm2 logs redmine-gitea-sync | grep ERROR
+```
+
+---
+
+## Getting Help
+
+### Collect Debug Information
+
+```bash
+# 1. Check server status
+curl http://localhost:3002/status
+
+# 2. Collect logs
+pm2 logs redmine-gitea-sync --lines 100 > debug.log
+
+# 3. Check configuration
+pm2 show redmine-gitea-sync
+
+# 4. Test connectivity
+curl -H "X-Redmine-API-Key: KEY" https://redmine.example.com/issues.json
+curl -H "Authorization: token TOKEN" https://gitea.example.com/api/v1/user
+
+# 5. Check process status
+pm2 list
+pm2 monit
+```
+
+### Report an Issue
+
+When reporting issues, include:
+
+1. Server status output
+2. Relevant logs (last 50-100 lines)
+3. Configuration (with sensitive data removed)
+4. Steps to reproduce
+5. Expected vs actual behavior
+
+### Additional Resources
+
+- [README.md](README.md) - Full documentation
+- [PM2_GUIDE.md](PM2_GUIDE.md) - PM2 management
+- [API_DOCUMENTATION.md](API_DOCUMENTATION.md) - API reference
+- [DEPLOYMENT.md](DEPLOYMENT.md) - Deployment guide
\ No newline at end of file