Intiial Commit.
Signed-off-by: Thomas Scott <tombomb122@noreply.localhost>
This commit is contained in:
commit
bb51054a35
|
|
@ -0,0 +1,465 @@
|
|||
# Redmine-Gitea Sync Server
|
||||
|
||||
A comprehensive bidirectional synchronization server between Redmine and Gitea issue tracking systems. This server maintains real-time synchronization of issues, comments, labels, milestones, and metadata between both platforms.
|
||||
|
||||
**Recommended Process Manager:** PM2 for production deployments with auto-restart, monitoring, and clustering capabilities.
|
||||
|
||||
## Features
|
||||
|
||||
### Core Synchronization
|
||||
- **Bidirectional sync**: Issues created or updated in either system automatically sync to the other
|
||||
- **Loop prevention**: Intelligent caching prevents infinite sync loops
|
||||
- **Issue mapping**: Maintains permanent links between Redmine and Gitea issues
|
||||
- **Status synchronization**: Maps status states between both systems
|
||||
|
||||
### Advanced Features
|
||||
- **Comments/Journals**: Preserves complete comment history
|
||||
- **Labels**: Syncs Redmine trackers, priorities, and categories as Gitea labels
|
||||
- **Milestones**: Syncs Redmine versions as Gitea milestones
|
||||
- **Metadata**: Syncs assignees, due dates, start dates, estimated hours
|
||||
- **Custom fields**: Optionally includes Redmine custom fields in descriptions
|
||||
- **Retry logic**: Automatic retry with exponential backoff for failed requests
|
||||
- **Comprehensive logging**: Multiple log levels with optional verbose mode
|
||||
|
||||
## Architecture
|
||||
|
||||
### Sync Flow
|
||||
|
||||
**Redmine → Gitea:**
|
||||
1. Redmine webhook triggers on issue update
|
||||
2. Server checks cache to prevent loops
|
||||
3. Searches for existing Gitea issue by Redmine ID
|
||||
4. Creates or updates Gitea issue with all metadata
|
||||
5. Syncs labels, milestones, and comments
|
||||
6. Adds entry to sync cache
|
||||
|
||||
**Gitea → Redmine:**
|
||||
1. Gitea webhook triggers on issue update
|
||||
2. Server checks cache to prevent loops
|
||||
3. Searches for existing Redmine issue by title or ID
|
||||
4. Creates or updates Redmine issue with all metadata
|
||||
5. Syncs comments as journals
|
||||
6. Adds entry to sync cache
|
||||
|
||||
### Cache Strategy
|
||||
- In-memory cache with configurable TTL (default: 30 seconds)
|
||||
- Prevents sync loops while allowing future edits
|
||||
- Separate caches for issues and comments
|
||||
- Automatic cleanup on expiry
|
||||
|
||||
### Error Handling
|
||||
- Automatic retry with exponential backoff
|
||||
- Comprehensive error logging
|
||||
- Graceful degradation on non-critical failures
|
||||
- Connection health checks
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
- Node.js 18+
|
||||
- npm or yarn
|
||||
- PM2 process manager (recommended)
|
||||
- Redmine instance with API access
|
||||
- Gitea instance with API access
|
||||
- Network connectivity between server and both platforms
|
||||
|
||||
### Setup
|
||||
|
||||
1. Clone the repository:
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd redmine-gitea-sync
|
||||
```
|
||||
|
||||
2. Install dependencies:
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
3. Install PM2 globally (if not already installed):
|
||||
```bash
|
||||
npm install -g pm2
|
||||
```
|
||||
|
||||
4. Configure environment variables:
|
||||
```bash
|
||||
# Edit the PM2 ecosystem config with your credentials
|
||||
nano ecosystem.config.cjs
|
||||
```
|
||||
|
||||
Update the `env` section with your Redmine and Gitea details.
|
||||
|
||||
5. Start the server with PM2:
|
||||
```bash
|
||||
# Start the server
|
||||
pm2 start ecosystem.config.cjs
|
||||
|
||||
# Save PM2 configuration
|
||||
pm2 save
|
||||
|
||||
# Setup auto-start on system boot
|
||||
pm2 startup
|
||||
```
|
||||
|
||||
For development with auto-reload:
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### Alternative: Start without PM2
|
||||
```bash
|
||||
npm start
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### PM2 Configuration (Recommended)
|
||||
|
||||
Edit `ecosystem.config.cjs` with your settings:
|
||||
|
||||
```javascript
|
||||
env: {
|
||||
NODE_ENV: 'production',
|
||||
PORT: 3002,
|
||||
|
||||
// Redmine Configuration
|
||||
REDMINE_API_URL: 'https://redmine.example.com',
|
||||
REDMINE_API_KEY: 'your_redmine_api_key',
|
||||
|
||||
// Gitea Configuration
|
||||
GITEA_API_URL: 'https://gitea.example.com',
|
||||
GITEA_TOKEN: 'your_gitea_token',
|
||||
GITEA_OWNER: 'your_gitea_username',
|
||||
|
||||
// Optional configurations
|
||||
CACHE_TTL: '30000',
|
||||
SYNC_LABELS: 'true',
|
||||
SYNC_MILESTONES: 'true',
|
||||
LOG_LEVEL: 'info',
|
||||
}
|
||||
```
|
||||
|
||||
After updating, restart PM2:
|
||||
```bash
|
||||
pm2 restart redmine-gitea-sync --update-env
|
||||
```
|
||||
|
||||
### Alternative: .env File Configuration
|
||||
|
||||
Create a `.env` file for environment-based configuration:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
|
||||
### Required Environment Variables
|
||||
|
||||
```env
|
||||
# Server Configuration
|
||||
PORT=3002
|
||||
|
||||
# Redmine Configuration
|
||||
REDMINE_API_URL=https://redmine.example.com
|
||||
REDMINE_API_KEY=your_redmine_api_key
|
||||
|
||||
# Gitea Configuration
|
||||
GITEA_API_URL=https://gitea.example.com
|
||||
GITEA_TOKEN=your_gitea_token
|
||||
GITEA_OWNER=your_gitea_username
|
||||
```
|
||||
|
||||
### Optional Environment Variables
|
||||
|
||||
```env
|
||||
# Timeouts (milliseconds)
|
||||
REDMINE_TIMEOUT=30000
|
||||
GITEA_TIMEOUT=30000
|
||||
|
||||
# Cache Configuration
|
||||
CACHE_TTL=30000
|
||||
|
||||
# Sync Features (true/false)
|
||||
SYNC_LABELS=true
|
||||
SYNC_MILESTONES=true
|
||||
SYNC_ATTACHMENTS=true
|
||||
SYNC_CUSTOM_FIELDS=true
|
||||
|
||||
# Retry Configuration
|
||||
RETRY_ATTEMPTS=3
|
||||
RETRY_DELAY=1000
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=info
|
||||
LOG_VERBOSE=false
|
||||
|
||||
# Project Mapping (JSON)
|
||||
PROJECT_MAPPING={"AI_Smuggling":"ai-smuggling","MyRepo":"my-project"}
|
||||
```
|
||||
|
||||
### Project Mapping
|
||||
|
||||
The `PROJECT_MAPPING` variable allows you to define custom mappings between Gitea repository names and Redmine project identifiers:
|
||||
|
||||
```json
|
||||
{
|
||||
"GiteaRepoName": "redmine-project-identifier",
|
||||
"AI_Smuggling": "ai-smuggling",
|
||||
"WebApp": "web-application"
|
||||
}
|
||||
```
|
||||
|
||||
If no mapping is provided, the server will automatically convert repository names:
|
||||
- Replace underscores with hyphens
|
||||
- Convert to lowercase
|
||||
- Remove special characters
|
||||
|
||||
## Webhook Configuration
|
||||
|
||||
### Redmine Webhook Setup
|
||||
|
||||
1. Navigate to: Administration → Settings → Repositories
|
||||
2. Add a new webhook with URL: `http://your-server:3002/webhook/redmine`
|
||||
3. Select events to trigger:
|
||||
- Issues created
|
||||
- Issues updated
|
||||
4. Save configuration
|
||||
|
||||
### Gitea Webhook Setup
|
||||
|
||||
1. Navigate to your repository → Settings → Webhooks
|
||||
2. Add webhook with:
|
||||
- Target URL: `http://your-server:3002/webhook/gitea`
|
||||
- HTTP Method: POST
|
||||
- POST Content Type: application/json
|
||||
- Trigger on: Issues, Issue Comment
|
||||
3. Save webhook
|
||||
|
||||
**Security Note**: For production deployments, consider:
|
||||
- Using HTTPS for webhook URLs
|
||||
- Implementing webhook signature verification
|
||||
- Restricting server access via firewall rules
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Health Check
|
||||
```
|
||||
GET /health
|
||||
```
|
||||
Returns server status and configuration information.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"uptime": 12345.67,
|
||||
"timestamp": "2025-12-17T01:49:47Z",
|
||||
"cache": {
|
||||
"size": 5,
|
||||
"ttl": 30000
|
||||
},
|
||||
"config": {
|
||||
"redmine": {
|
||||
"url": "https://redmine.example.com",
|
||||
"connected": true
|
||||
},
|
||||
"gitea": {
|
||||
"url": "https://gitea.example.com",
|
||||
"owner": "username",
|
||||
"connected": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Status Check
|
||||
```
|
||||
GET /status
|
||||
```
|
||||
Tests connectivity to both Redmine and Gitea.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"server": "running",
|
||||
"uptime": 12345.67,
|
||||
"cache": {
|
||||
"size": 5,
|
||||
"ttl": 30000
|
||||
},
|
||||
"connections": {
|
||||
"redmine": "connected",
|
||||
"gitea": "connected"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Clear Cache
|
||||
```
|
||||
POST /cache/clear
|
||||
```
|
||||
Manually clears the sync cache.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"message": "Cache cleared",
|
||||
"timestamp": "2025-12-17T01:49:47Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Status Mapping
|
||||
|
||||
### Redmine → Gitea
|
||||
| Redmine Status | Gitea State |
|
||||
|----------------|-------------|
|
||||
| New (1) | open |
|
||||
| In Progress (2) | open |
|
||||
| Resolved (3) | open |
|
||||
| Feedback (4) | open |
|
||||
| Closed (5) | closed |
|
||||
| Rejected (6) | closed |
|
||||
|
||||
### Gitea → Redmine
|
||||
| Gitea State | Redmine Status |
|
||||
|-------------|----------------|
|
||||
| open | In Progress (2) |
|
||||
| closed | Closed (5) |
|
||||
|
||||
## Label Mapping
|
||||
|
||||
The server automatically creates and syncs labels based on Redmine metadata:
|
||||
|
||||
- **Trackers**: bug, feature, support
|
||||
- **Priorities**: Low Priority, Normal Priority, High Priority, Urgent, Immediate
|
||||
- **Categories**: Prefixed with `category:`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue: Webhooks not triggering**
|
||||
- Verify webhook URLs are accessible from Redmine/Gitea servers
|
||||
- Check firewall rules
|
||||
- Verify webhook is enabled and event types are selected
|
||||
- Check server logs for incoming requests
|
||||
|
||||
**Issue: Sync loops occurring**
|
||||
- Verify cache TTL is sufficient (minimum 30 seconds recommended)
|
||||
- Check that both webhooks are not triggering simultaneously
|
||||
- Review logs for cache hit/miss patterns
|
||||
|
||||
**Issue: Issues not syncing**
|
||||
- Verify API credentials are correct
|
||||
- Check project mapping configuration
|
||||
- Ensure projects exist in both systems
|
||||
- Review logs for error messages
|
||||
|
||||
**Issue: Comments not appearing**
|
||||
- Verify journal/comment webhooks are enabled
|
||||
- Check that journal notes are not empty
|
||||
- Review cache for recent sync entries
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Enable verbose logging for detailed debugging:
|
||||
```env
|
||||
LOG_LEVEL=debug
|
||||
LOG_VERBOSE=true
|
||||
```
|
||||
|
||||
This will output:
|
||||
- All API requests and responses
|
||||
- Cache operations
|
||||
- Detailed error stack traces
|
||||
- Sync decision logic
|
||||
|
||||
### Log Levels
|
||||
|
||||
- `debug`: Detailed information for debugging
|
||||
- `info`: General operational messages (default)
|
||||
- `warn`: Warning messages for recoverable issues
|
||||
- `error`: Error messages for failures
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Recommendations
|
||||
- Deploy server close to both Redmine and Gitea instances
|
||||
- Use adequate cache TTL to prevent excessive API calls
|
||||
- Monitor memory usage for large installations
|
||||
- Consider rate limiting for high-traffic scenarios
|
||||
|
||||
### Scalability
|
||||
- The server maintains in-memory cache (suitable for single instance)
|
||||
- For multi-instance deployments, consider Redis for shared cache
|
||||
- Current implementation handles up to 1000 issues per repository efficiently
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **API Keys**: Store credentials securely, never commit to version control
|
||||
2. **Network**: Use HTTPS for all webhook URLs
|
||||
3. **Access Control**: Restrict server access to necessary IPs only
|
||||
4. **Validation**: Webhook signature verification recommended for production
|
||||
5. **Monitoring**: Implement logging and monitoring for suspicious activity
|
||||
|
||||
## Development
|
||||
|
||||
### Running Tests
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
|
||||
### Code Structure
|
||||
```
|
||||
.
|
||||
├── server.js # Main server file
|
||||
├── .env # Environment configuration
|
||||
├── .env.example # Example configuration
|
||||
├── package.json # Dependencies
|
||||
└── README.md # Documentation
|
||||
```
|
||||
|
||||
### Contributing
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Make your changes
|
||||
4. Add tests if applicable
|
||||
5. Submit a pull request
|
||||
|
||||
## License
|
||||
|
||||
Please see License file.
|
||||
|
||||
## Support
|
||||
|
||||
For issues, questions, or contributions:
|
||||
- Create an issue in the repository
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 2.0.1 (Latest)
|
||||
- Fixed label to field mapping (Gitea → Redmine) - tracker and priority now parse correctly from labels
|
||||
- Fixed Redmine comments not syncing to Gitea - journals now always process on updates
|
||||
- Fixed date format validation errors - proper conversion between ISO 8601 and YYYY-MM-DD formats
|
||||
- Fixed label handling 422 errors - labels now set separately with proper ID resolution
|
||||
- Enhanced error logging - 422 errors now show detailed validation messages
|
||||
- Improved journal processing with debug logging
|
||||
- See [FIX_SUMMARY.md](FIX_SUMMARY.md) for detailed information
|
||||
|
||||
### Version 2.0.0
|
||||
- Complete rewrite with comprehensive API support
|
||||
- Added PM2 process manager support with ecosystem config
|
||||
- Added label synchronization
|
||||
- Added milestone synchronization
|
||||
- Enhanced error handling with retry logic
|
||||
- Improved logging system with multiple levels
|
||||
- Added health check and status endpoints
|
||||
- Added custom field support
|
||||
- Better project mapping configuration
|
||||
- Complete documentation suite including PM2 management guide
|
||||
|
||||
### Version 1.0.0
|
||||
- Initial release
|
||||
- Basic bidirectional sync
|
||||
- Loop prevention
|
||||
- Comment/journal sync
|
||||
Loading…
Reference in New Issue