Cron Setup

This guide explains how to configure Symfony Messenger for production environments and optimal translation processing.

Overview

The TMGMT Lara Translate module uses Symfony Messenger for asynchronous translation processing. Unlike the old Queue API system, Symfony Messenger provides more flexibility in transport options and deployment scenarios.

Messenger Configuration

Default Setup (Development)

The module works out-of-the-box with Drupal's default configuration: - Database transport: Messages stored in Drupal database - Single worker: Processes messages during cron runs - Automatic retry: Built-in exponential backoff - No external dependencies: Works with default Drupal installation

Production Transport Options

Redis Transport (Recommended)

For high-performance production sites:

Benefits: - High speed: In-memory message processing - Scalability: Handles large message volumes - Persistence: Survives server restarts - Reliability: Redis clustering support

Setup Requirements: - Redis server accessible from Drupal server - PHP Redis extension installed - Messenger configuration for Redis transport

Configuration:

# settings.php
$settings['messenger'] = [
  'transports' => [
    'async' => 'redis://localhost:6379/messages'
  ],
  'routing' => [
    'Drupal\tmgmt_laratranslate\Messenger\TranslationJobMessage' => ['async']
  ]
];

Database Transport (Default)

For smaller sites or development:

Benefits: - No external dependencies - Simple setup - Integrated with Drupal: Uses existing database

Configuration:

# Uses default database transport
$settings['messenger'] = [
  'transports' => [
    'async' => 'doctrine://default'
  ]
];

Custom Transports

Symfony Messenger supports various transports:

• AMQP: RabbitMQ message queues
• SQS: Amazon Simple Queue Service
• Kafka: High-throughput distributed messaging
• FileSystem: File-based message storage

Worker Configuration

Cron-Based Processing

Traditional approach using Drupal cron:

# Default cron configuration
$settings['messenger']['consumers'] = [
    'async' => [
        'worker' => 'drush messenger:consume async',
        'time_limit' => 60,
        'memory_limit' => '256M',
    ],
];

Dedicated Worker Processes

For high-volume sites, run dedicated workers:

# Start long-running worker process
drush messenger:consume async --time-limit=3600 --memory-limit=512M

# Run multiple workers in background
nohup drush messenger:consume async --worker-id=1 &
nohup drush messenger:consume async --worker-id=2 &
nohup drush messenger:consume async --worker-id=3 &

Process Supervisor

Use process manager for reliable worker management:

# /etc/supervisor/conf.d/tmgmt-laratranslate.conf
[program:tmgmt-laratranslate-worker]
command=/usr/bin/drush messenger:consume async --time-limit=3600
process_name=%(program_name)s_%(process_num)02d
numprocs=4
autostart=true
autorestart=true
user=www-data
stdout_logfile=/var/log/tmgmt-worker.log

Performance Optimization

Worker Scaling

Single Worker Performance

# Monitor single worker performance
drush messenger:consume async --limit=100 --time-limit=300

# Check processing rate
time drush messenger:consume async --limit=1000 | grep -c "messages processed"

Multiple Workers

# Start 4 workers for parallel processing
for i in {1..4}; do
    drush messenger:consume async --worker-id=$i --time-limit=3600 &
done

# Monitor all workers
watch "drush messenger:stats"

Memory Management

# Set appropriate memory limits
drush messenger:consume async --memory-limit=512M

# Monitor memory usage
drush status --format=json | jq '.["php-memory"]'

Throughput Optimization

Message Batching

Process multiple messages per cycle:

# Process 50 messages per batch
drush messenger:consume async --limit=50

# Continuous processing
drush messenger:consume async --sleep=1

Queue Depth Monitoring

# Check message queue depth
drush messenger:info

# For Redis transport
redis-cli llen tmgmt_laratranslate:messages

Environment Configuration

Development Environment

# development.services.yml
services:
  Drupal\tmgmt_laratranslate\MessageHandler\TranslationJobMessageHandler:
    arguments:
      - '@entity_type.manager'
      - '@logger.channel.tmgmt_laratranslate'
    tags:
      - { name: 'messenger.message_handler' }

Production Environment

# production.services.yml
services:
  Drupal\tmgmt_laratranslate\MessageHandler\TranslationJobMessageHandler:
    arguments:
      - '@entity_type.manager'
      - '@logger.channel.tmgmt_laratranslate'
    tags:
      - { name: 'messenger.message_handler' }

Staging Environment

# staging.services.yml
services:
  Drupal\tmgmt_laratranslate\MessageHandler\TranslationJobMessageHandler:
    arguments:
      - '@entity_type.manager'
      - '@logger.channel.tmgmt_laratranslate'
    tags:
      - { name: 'messenger.message_handler' }

Monitoring and Observability

Health Checks

# Check Messenger status
drush messenger:info

# Test message processing
drush messenger:consume async --limit=1 --verbose

# Check worker processes
ps aux | grep "drush messenger:consume"

Performance Metrics

# Message processing statistics
drush messenger:stats

# Queue depth over time
watch -n 60 "drush messenger:info | grep 'pending'"

# Worker resource usage
top -p $(pgrep -f "drush messenger")

Log Monitoring

# Monitor translation logs
drush watchdog:show --type=tmgmt_laratranslate --follow

# Filter by error level
drush watchdog:show --severity=error --type=tmgmt_laratranslate

# Check messenger-specific logs
drush watchdog:show --type=messenger --limit=20

Deployment Strategies

Single Server Deployment

Basic Setup

• One Drupal instance processing messages
• Suitable for small to medium sites
• Uses default database or Redis transport
• Simple monitoring and maintenance

Load Balancing

# Multiple Drupal instances behind load balancer
$instance_id = $_SERVER['INSTANCE_ID'] ?? 'default';

$settings['messenger'] = [
  'transports' => [
    'async' => "redis://redis-cluster:6379/messages?instance={$instance_id}"
  ]
];

Multi-Server Deployment

Horizontal Scaling

Multiple Drupal instances sharing message queue:

Redis Shared Queue: - All instances connect to same Redis cluster - Messages distributed across available workers - Automatic load balancing

Configuration:

# Instance-specific configuration
$instance_config = [
    'server-1' => 'redis://redis-cluster:6379/messages',
    'server-2' => 'redis://redis-cluster:6379/messages',
    'server-3' => 'redis://redis-cluster:6379/messages',
];

$settings['messenger'] = [
  'transports' => [
    'async' => $instance_config[$_SERVER['INSTANCE_ID']]
  ]
];

Troubleshooting

Common Messenger Issues

Messages Not Processing

Symptoms: Messages queued but not consumed

Checks:

# Verify messenger configuration
drush config:get messenger.settings

# Check message handler registration
drush messenger:list-handlers | grep TranslationJobMessageHandler

# Check queue depth
drush messenger:info

# Test manual processing
drush messenger:consume async --limit=1 --verbose

Worker Performance Issues

Symptoms: Slow message processing

Checks:

# Monitor worker memory usage
drush messenger:consume async --memory-limit=512M --verbose

# Check processing time per message
time drush messenger:consume async --limit=10

# Profile worker performance
drush messenger:consume async --limit=100 --profile

Transport Connection Issues

Symptoms: Connection failures to Redis/database

Checks:

# Test Redis connection
redis-cli -h localhost -p 6379 ping

# Test database connection
drush sql:query "SELECT 1" --database=drupal

# Check transport configuration
drush config:get messenger.transports

Best Practices

Production Setup

1. Use Redis transport for better performance
2. Configure multiple workers for high throughput
3. Set memory limits appropriate for content size
4. Monitor queue depth to prevent backlogs
5. Use process supervisor for reliable workers

Security Considerations

1. Secure Redis connections with passwords
2. Network isolation for message queue
3. Access controls for worker processes
4. Audit logs for suspicious activity

Maintenance Procedures

1. Regular log rotation for worker processes
2. Queue monitoring with automated alerts
3. Performance tuning based on usage patterns
4. Backup procedures for messenger configuration

Integration with Other Systems

Monitoring Integration

# Export metrics to monitoring system
drush messenger:stats --format=json | curl -X POST http://monitoring.example.com/metrics

# Alert on high queue depth
if [ $(drush messenger:info | grep 'pending:' | awk '{print $2}') -gt 1000 ]; then
    curl -X POST -H "Content-Type: application/json" \
         -d '{"alert": "High message queue depth", "count": '$(drush messenger:info | grep "pending:" | awk '{print $2}')'}' \
         http://alerting.example.com/webhook
fi

CI/CD Integration

# .gitlab-ci.yml for messenger testing
test:messenger:
  script:
    - drush messenger:consume async --limit=1
    - drush messenger:stats
  artifacts:
    reports:
      junit: reports/

Related Documentation

• Architecture Guide - Symfony Messenger implementation
• Drush Commands Guide - Command-line tools
• Troubleshooting Guide - Common issues and solutions