docker2pbs/README.md

Docker2PBS

Docker Compose to Proxmox Backup Server integration tool. Automatically backup Docker services with their volumes and configuration to Proxmox Backup Server.

Features

• ✅ Docker Compose Integration - Reads service configuration directly from docker-compose.yml
• ✅ Volume Detection - Automatically identifies bind mounts, named volumes, and external volumes
• ✅ Network Analysis - Parses network configuration including external networks
• ✅ Safe Operations - Stops service before backup, restarts after completion
• ✅ Separate Archives - Creates individual .pxar archives for each volume
• ✅ Selective Restore - Restore individual volumes or complete service
• ✅ Dry-run Mode - Test configuration without executing operations
• ✅ PBS Integration - Direct integration with Proxmox Backup Server
• ✅ Authentication Support - Supports PBS username/password/fingerprint authentication

Requirements

• Python 3.6+
• PyYAML
• Docker and docker-compose
• proxmox-backup-client installed and configured

Installation

# Clone or download the script
git clone <repository_url>
cd docker2pbs

# Install dependencies
pip install -r requirements.txt

# Make script executable
chmod +x docker2pbs.py

Quick Start

# Basic backup
./docker2pbs.py docker-compose.yml web --pbs-repository user@pbs.example.com:datastore

# Test configuration first
./docker2pbs.py docker-compose.yml web --pbs-repository user@pbs.example.com:datastore --dry-run

# With authentication
./docker2pbs.py docker-compose.yml database \
    --pbs-repository user@pbs.example.com:datastore \
    --pbs-username backup-user \
    --pbs-password secret123

Usage

./docker2pbs.py <compose_file> <service_name> --pbs-repository <repository> [options]

Arguments:
  compose_file          Path to docker-compose.yaml file
  service_name         Name of the service to backup

Required Options:
  --pbs-repository     PBS repository (format: user@host:datastore)

Optional:
  --pbs-username       PBS username for authentication
  --pbs-password       PBS password for authentication  
  --pbs-fingerprint    PBS server fingerprint for verification
  --dry-run           Show what would be done without executing
  --help              Show help message

How It Works

Backup Process

1. Configuration Parsing

   • Reads docker-compose.yml file
   • Extracts service configuration (volumes, networks)
   • Identifies volume types (bind mounts vs named volumes)
   • Detects external volumes and networks

2. Service Management

   • Stops the target service using docker-compose stop
   • Ensures data consistency during backup

3. Backup Creation

   • Creates separate .pxar archive for each volume
   • Uses descriptive archive names (e.g., volume_db_data.pxar, bind_dot_html.pxar)
   • Uploads to Proxmox Backup Server

4. Service Recovery

   • Restarts the service using docker-compose up -d
   • Ensures service availability is restored

Archive Naming Convention

Volume Type	Source Example	Archive Name
Named volume	db_data	volume_db_data.pxar
Bind mount	./html	bind_dot_html.pxar
Bind mount	/app/data	bind__app_data.pxar
Bind mount	../config	bind_dotdot_config.pxar

Dry-run Mode

Use --dry-run to preview operations:

• Shows detailed service configuration
• Lists all volumes that would be backed up
• Displays archive names that would be created
• Shows exact commands that would be executed
• No actual operations performed

Examples

Basic Web Service Backup

# docker-compose.yml
services:
  web:
    image: nginx:latest
    volumes:
      - ./html:/var/www/html
      - ./config:/etc/nginx/conf.d
    networks:
      - webnet

# Backup the web service
./docker2pbs.py docker-compose.yml web --pbs-repository backup@pbs.local:webapps

# Creates these archives:
# - bind_dot_html.pxar (contains ./html content)
# - bind_dot_config.pxar (contains ./config content)

Database Service with Named Volumes

# docker-compose.yml
services:
  database:
    image: postgres:13
    volumes:
      - db_data:/var/lib/postgresql/data
      - db_backups:/backups
    environment:
      POSTGRES_PASSWORD: secret

volumes:
  db_data:
  db_backups:
    external: true

# Backup database service
./docker2pbs.py docker-compose.yml database \
    --pbs-repository dba@pbs.local:databases \
    --pbs-username backup-user

# Creates these archives:
# - volume_db_data.pxar (PostgreSQL data directory)
# - volume_db_backups.pxar (backup directory)

Restore Operations

See restore_example.md for detailed restore procedures.

Quick Restore Example

# List available backups
proxmox-backup-client list --repository user@pbs.example.com:datastore

# Restore specific volume
proxmox-backup-client restore \
    --repository user@pbs.example.com:datastore \
    host/web/2024-01-15_10-30-45 \
    volume_db_data.pxar /var/lib/docker/volumes/db_data/_data

Configuration

PBS Client Setup

Ensure proxmox-backup-client is installed and configured:

# Install PBS client (Debian/Ubuntu)
apt install proxmox-backup-client

# Configure repository (optional)
export PBS_REPOSITORY=user@pbs.example.com:datastore
export PBS_PASSWORD=your_password

Supported Volume Types

• ✅ Bind Mounts - Local directories mounted into containers
• ✅ Named Volumes - Docker-managed volumes
• ✅ External Volumes - Pre-existing volumes managed outside compose
• ❌ Tmpfs Volumes - Memory-based volumes (not backed up)
• ❌ Anonymous Volumes - Unnamed volumes (not backed up)

Supported Network Types

• ✅ Default Networks - Docker compose default networks
• ✅ Named Networks - Custom networks defined in compose
• ✅ External Networks - Pre-existing networks
• ✅ Bridge Networks - Standard bridge networks
• ✅ Overlay Networks - Multi-host networks

Security Considerations

• PBS credentials can be provided via command line or environment variables
• Avoid hardcoding passwords in scripts
• Use PBS fingerprint verification for secure connections
• Consider using PBS tokens instead of passwords
• Ensure backup user has minimal required permissions

Troubleshooting

Common Issues

1. Service fails to stop

   # Check if service is running
   docker-compose ps
   
   # Manually stop if needed
   docker-compose stop service_name
   

2. Volume path not found

   # Check volume exists
   docker volume ls
   docker volume inspect volume_name
   
   # For bind mounts, verify path exists
   ls -la /path/to/bind/mount
   

3. PBS connection fails

   # Test PBS connectivity
   proxmox-backup-client list --repository user@pbs.example.com:datastore
   
   # Verify credentials
   echo $PBS_PASSWORD
   

4. Permission issues

   # Ensure script is executable
   chmod +x docker2pbs.py
   
   # Check Docker permissions
   docker ps
   

Debug Mode

For verbose output, modify the script to enable debug logging or run with:

# Show all executed commands
bash -x docker2pbs.py docker-compose.yml service --pbs-repository repo --dry-run

Contributing

1. Fork the repository
2. Create feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• Create issues for bugs or feature requests
• Check existing documentation in /docs directory
• Review example configurations in /examples directory

Related Projects

• Proxmox Backup Server
• Docker Compose
• proxmox-backup-client