Redmine-Gitea-Sync/README.md

# Redmine-Gitea Sync Server

A comprehensive bidirectional synchronization server between Redmine and Gitea issue tracking systems. This server maintains real-time synchronization of issues, comments, labels, milestones, and metadata between both platforms.

**Recommended Process Manager:** PM2 for production deployments with auto-restart, monitoring, and clustering capabilities.

## Features

### Core Synchronization
- **Bidirectional sync**: Issues created or updated in either system automatically sync to the other
- **Loop prevention**: Intelligent caching prevents infinite sync loops
- **Issue mapping**: Maintains permanent links between Redmine and Gitea issues
- **Status synchronization**: Maps status states between both systems

### Advanced Features
- **Comments/Journals**: Preserves complete comment history
- **Labels**: Syncs Redmine trackers, priorities, and categories as Gitea labels
- **Milestones**: Syncs Redmine versions as Gitea milestones
- **Metadata**: Syncs assignees, due dates, start dates, estimated hours
- **Custom fields**: Optionally includes Redmine custom fields in descriptions
- **Retry logic**: Automatic retry with exponential backoff for failed requests
- **Comprehensive logging**: Multiple log levels with optional verbose mode

## Architecture

### Sync Flow

**Redmine → Gitea:**
1. Redmine webhook triggers on issue update
2. Server checks cache to prevent loops
3. Searches for existing Gitea issue by Redmine ID
4. Creates or updates Gitea issue with all metadata
5. Syncs labels, milestones, and comments
6. Adds entry to sync cache

**Gitea → Redmine:**
1. Gitea webhook triggers on issue update
2. Server checks cache to prevent loops
3. Searches for existing Redmine issue by title or ID
4. Creates or updates Redmine issue with all metadata
5. Syncs comments as journals
6. Adds entry to sync cache

### Cache Strategy
- In-memory cache with configurable TTL (default: 30 seconds)
- Prevents sync loops while allowing future edits
- Separate caches for issues and comments
- Automatic cleanup on expiry

### Error Handling
- Automatic retry with exponential backoff
- Comprehensive error logging
- Graceful degradation on non-critical failures
- Connection health checks

## Installation

### Prerequisites
- Node.js 18+ 
- npm or yarn
- PM2 process manager (recommended)
- Redmine instance with API access
- Gitea instance with API access
- Network connectivity between server and both platforms

### Setup

1. Clone the repository:
```bash
git clone <repository-url>
cd redmine-gitea-sync
```

2. Install dependencies:
```bash
npm install
```

3. Install PM2 globally (if not already installed):
```bash
npm install -g pm2
```

4. Configure environment variables:
```bash
# Edit the PM2 ecosystem config with your credentials
nano ecosystem.config.cjs
```

Update the `env` section with your Redmine and Gitea details.

5. Start the server with PM2:
```bash
# Start the server
pm2 start ecosystem.config.cjs

# Save PM2 configuration
pm2 save

# Setup auto-start on system boot
pm2 startup
```

For development with auto-reload:
```bash
npm run dev
```

### Alternative: Start without PM2
```bash
npm start
```

## Configuration

### PM2 Configuration (Recommended)

Edit `ecosystem.config.cjs` with your settings:

```javascript
env: {
  NODE_ENV: 'production',
  PORT: 3002,
  
  // Redmine Configuration
  REDMINE_API_URL: 'https://redmine.example.com',
  REDMINE_API_KEY: 'your_redmine_api_key',
  
  // Gitea Configuration
  GITEA_API_URL: 'https://gitea.example.com',
  GITEA_TOKEN: 'your_gitea_token',
  GITEA_OWNER: 'your_gitea_username',
  
  // Optional configurations
  CACHE_TTL: '30000',
  SYNC_LABELS: 'true',
  SYNC_MILESTONES: 'true',
  LOG_LEVEL: 'info',
}
```

After updating, restart PM2:
```bash
pm2 restart redmine-gitea-sync --update-env
```

### Alternative: .env File Configuration

Create a `.env` file for environment-based configuration:

```bash
cp .env.example .env
nano .env
```

### Required Environment Variables

```env
# Server Configuration
PORT=3002

# Redmine Configuration
REDMINE_API_URL=https://redmine.example.com
REDMINE_API_KEY=your_redmine_api_key

# Gitea Configuration
GITEA_API_URL=https://gitea.example.com
GITEA_TOKEN=your_gitea_token
GITEA_OWNER=your_gitea_username
```

### Optional Environment Variables

```env
# Timeouts (milliseconds)
REDMINE_TIMEOUT=30000
GITEA_TIMEOUT=30000

# Cache Configuration
CACHE_TTL=30000

# Sync Features (true/false)
SYNC_LABELS=true
SYNC_MILESTONES=true
SYNC_ATTACHMENTS=true
SYNC_CUSTOM_FIELDS=true

# Retry Configuration
RETRY_ATTEMPTS=3
RETRY_DELAY=1000

# Logging
LOG_LEVEL=info
LOG_VERBOSE=false

# Project Mapping (JSON)
PROJECT_MAPPING={"AI_Smuggling":"ai-smuggling","MyRepo":"my-project"}
```

### Project Mapping

The `PROJECT_MAPPING` variable allows you to define custom mappings between Gitea repository names and Redmine project identifiers:

```json
{
  "GiteaRepoName": "redmine-project-identifier",
  "AI_Smuggling": "ai-smuggling",
  "WebApp": "web-application"
}
```

If no mapping is provided, the server will automatically convert repository names:
- Replace underscores with hyphens
- Convert to lowercase
- Remove special characters

## Webhook Configuration

### Redmine Webhook Setup

1. Navigate to: Administration → Settings → Repositories
2. Add a new webhook with URL: `http://your-server:3002/webhook/redmine`
3. Select events to trigger:
   - Issues created
   - Issues updated
4. Save configuration

### Gitea Webhook Setup

1. Navigate to your repository → Settings → Webhooks
2. Add webhook with:
   - Target URL: `http://your-server:3002/webhook/gitea`
   - HTTP Method: POST
   - POST Content Type: application/json
   - Trigger on: Issues, Issue Comment
3. Save webhook

**Security Note**: For production deployments, consider:
- Using HTTPS for webhook URLs
- Implementing webhook signature verification
- Restricting server access via firewall rules

## API Endpoints

### Health Check
```
GET /health
```
Returns server status and configuration information.

**Response:**
```json
{
  "status": "ok",
  "uptime": 12345.67,
  "timestamp": "2025-12-17T01:49:47Z",
  "cache": {
    "size": 5,
    "ttl": 30000
  },
  "config": {
    "redmine": {
      "url": "https://redmine.example.com",
      "connected": true
    },
    "gitea": {
      "url": "https://gitea.example.com",
      "owner": "username",
      "connected": true
    }
  }
}
```

### Status Check
```
GET /status
```
Tests connectivity to both Redmine and Gitea.

**Response:**
```json
{
  "server": "running",
  "uptime": 12345.67,
  "cache": {
    "size": 5,
    "ttl": 30000
  },
  "connections": {
    "redmine": "connected",
    "gitea": "connected"
  }
}
```

### Clear Cache
```
POST /cache/clear
```
Manually clears the sync cache.

**Response:**
```json
{
  "message": "Cache cleared",
  "timestamp": "2025-12-17T01:49:47Z"
}
```

## Status Mapping

### Redmine → Gitea
| Redmine Status | Gitea State |
|----------------|-------------|
| New (1) | open |
| In Progress (2) | open |
| Resolved (3) | open |
| Feedback (4) | open |
| Closed (5) | closed |
| Rejected (6) | closed |

### Gitea → Redmine
| Gitea State | Redmine Status |
|-------------|----------------|
| open | In Progress (2) |
| closed | Closed (5) |

## Label Mapping

The server automatically creates and syncs labels based on Redmine metadata:

- **Trackers**: bug, feature, support
- **Priorities**: Low Priority, Normal Priority, High Priority, Urgent, Immediate
- **Categories**: Prefixed with `category:`

## Troubleshooting

### Common Issues

**Issue: Webhooks not triggering**
- Verify webhook URLs are accessible from Redmine/Gitea servers
- Check firewall rules
- Verify webhook is enabled and event types are selected
- Check server logs for incoming requests

**Issue: Sync loops occurring**
- Verify cache TTL is sufficient (minimum 30 seconds recommended)
- Check that both webhooks are not triggering simultaneously
- Review logs for cache hit/miss patterns

**Issue: Issues not syncing**
- Verify API credentials are correct
- Check project mapping configuration
- Ensure projects exist in both systems
- Review logs for error messages

**Issue: Comments not appearing**
- Verify journal/comment webhooks are enabled
- Check that journal notes are not empty
- Review cache for recent sync entries

### Debug Mode

Enable verbose logging for detailed debugging:
```env
LOG_LEVEL=debug
LOG_VERBOSE=true
```

This will output:
- All API requests and responses
- Cache operations
- Detailed error stack traces
- Sync decision logic

### Log Levels

- `debug`: Detailed information for debugging
- `info`: General operational messages (default)
- `warn`: Warning messages for recoverable issues
- `error`: Error messages for failures

## Performance Considerations

### Recommendations
- Deploy server close to both Redmine and Gitea instances
- Use adequate cache TTL to prevent excessive API calls
- Monitor memory usage for large installations
- Consider rate limiting for high-traffic scenarios

### Scalability
- The server maintains in-memory cache (suitable for single instance)
- For multi-instance deployments, consider Redis for shared cache
- Current implementation handles up to 1000 issues per repository efficiently

## Security Best Practices

1. **API Keys**: Store credentials securely, never commit to version control
2. **Network**: Use HTTPS for all webhook URLs
3. **Access Control**: Restrict server access to necessary IPs only
4. **Validation**: Webhook signature verification recommended for production
5. **Monitoring**: Implement logging and monitoring for suspicious activity

## Development

### Running Tests
```bash
npm test
```

### Code Structure
```
.
├── server.js           # Main server file
├── .env               # Environment configuration
├── .env.example       # Example configuration
├── package.json       # Dependencies
└── README.md          # Documentation
```

### Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

## License

Please see License file. 

## Support

For issues, questions, or contributions:
- Create an issue in the repository

## Changelog

### Version 2.0.1 (Latest)
- Fixed label to field mapping (Gitea → Redmine) - tracker and priority now parse correctly from labels
- Fixed Redmine comments not syncing to Gitea - journals now always process on updates
- Fixed date format validation errors - proper conversion between ISO 8601 and YYYY-MM-DD formats
- Fixed label handling 422 errors - labels now set separately with proper ID resolution
- Enhanced error logging - 422 errors now show detailed validation messages
- Improved journal processing with debug logging
- See [FIX_SUMMARY.md](FIX_SUMMARY.md) for detailed information

### Version 2.0.0
- Complete rewrite with comprehensive API support
- Added PM2 process manager support with ecosystem config
- Added label synchronization
- Added milestone synchronization
- Enhanced error handling with retry logic
- Improved logging system with multiple levels
- Added health check and status endpoints
- Added custom field support
- Better project mapping configuration
- Complete documentation suite including PM2 management guide

### Version 1.0.0
- Initial release
- Basic bidirectional sync
- Loop prevention
- Comment/journal sync