Add comprehensive project documentation
- Complete README with features, installation, and usage - Docker compose integration examples - PBS authentication and configuration guide - Troubleshooting section with common issues - Archive naming convention documentation
This commit is contained in:
Radosław Gierwiało
298
README.md
Normal file
298
README.md
Normal file
@@ -0,0 +1,298 @@
|
||||
# Docker2PBS
|
||||
|
||||
Docker Compose to Proxmox Backup Server integration tool. Automatically backup Docker services with their volumes and configuration to Proxmox Backup Server.
|
||||
|
||||
## Features
|
||||
|
||||
- ✅ **Docker Compose Integration** - Reads service configuration directly from `docker-compose.yml`
|
||||
- ✅ **Volume Detection** - Automatically identifies bind mounts, named volumes, and external volumes
|
||||
- ✅ **Network Analysis** - Parses network configuration including external networks
|
||||
- ✅ **Safe Operations** - Stops service before backup, restarts after completion
|
||||
- ✅ **Separate Archives** - Creates individual `.pxar` archives for each volume
|
||||
- ✅ **Selective Restore** - Restore individual volumes or complete service
|
||||
- ✅ **Dry-run Mode** - Test configuration without executing operations
|
||||
- ✅ **PBS Integration** - Direct integration with Proxmox Backup Server
|
||||
- ✅ **Authentication Support** - Supports PBS username/password/fingerprint authentication
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.6+
|
||||
- PyYAML
|
||||
- Docker and docker-compose
|
||||
- `proxmox-backup-client` installed and configured
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Clone or download the script
|
||||
git clone <repository_url>
|
||||
cd docker2pbs
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Make script executable
|
||||
chmod +x docker2pbs.py
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Basic backup
|
||||
./docker2pbs.py docker-compose.yml web --pbs-repository user@pbs.example.com:datastore
|
||||
|
||||
# Test configuration first
|
||||
./docker2pbs.py docker-compose.yml web --pbs-repository user@pbs.example.com:datastore --dry-run
|
||||
|
||||
# With authentication
|
||||
./docker2pbs.py docker-compose.yml database \
|
||||
--pbs-repository user@pbs.example.com:datastore \
|
||||
--pbs-username backup-user \
|
||||
--pbs-password secret123
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
./docker2pbs.py <compose_file> <service_name> --pbs-repository <repository> [options]
|
||||
|
||||
Arguments:
|
||||
compose_file Path to docker-compose.yaml file
|
||||
service_name Name of the service to backup
|
||||
|
||||
Required Options:
|
||||
--pbs-repository PBS repository (format: user@host:datastore)
|
||||
|
||||
Optional:
|
||||
--pbs-username PBS username for authentication
|
||||
--pbs-password PBS password for authentication
|
||||
--pbs-fingerprint PBS server fingerprint for verification
|
||||
--dry-run Show what would be done without executing
|
||||
--help Show help message
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### Backup Process
|
||||
|
||||
1. **Configuration Parsing**
|
||||
- Reads `docker-compose.yml` file
|
||||
- Extracts service configuration (volumes, networks)
|
||||
- Identifies volume types (bind mounts vs named volumes)
|
||||
- Detects external volumes and networks
|
||||
|
||||
2. **Service Management**
|
||||
- Stops the target service using `docker-compose stop`
|
||||
- Ensures data consistency during backup
|
||||
|
||||
3. **Backup Creation**
|
||||
- Creates separate `.pxar` archive for each volume
|
||||
- Uses descriptive archive names (e.g., `volume_db_data.pxar`, `bind_dot_html.pxar`)
|
||||
- Uploads to Proxmox Backup Server
|
||||
|
||||
4. **Service Recovery**
|
||||
- Restarts the service using `docker-compose up -d`
|
||||
- Ensures service availability is restored
|
||||
|
||||
### Archive Naming Convention
|
||||
|
||||
| Volume Type | Source Example | Archive Name |
|
||||
|-------------|----------------|--------------|
|
||||
| Named volume | `db_data` | `volume_db_data.pxar` |
|
||||
| Bind mount | `./html` | `bind_dot_html.pxar` |
|
||||
| Bind mount | `/app/data` | `bind__app_data.pxar` |
|
||||
| Bind mount | `../config` | `bind_dotdot_config.pxar` |
|
||||
|
||||
### Dry-run Mode
|
||||
|
||||
Use `--dry-run` to preview operations:
|
||||
- Shows detailed service configuration
|
||||
- Lists all volumes that would be backed up
|
||||
- Displays archive names that would be created
|
||||
- Shows exact commands that would be executed
|
||||
- **No actual operations performed**
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Web Service Backup
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
web:
|
||||
image: nginx:latest
|
||||
volumes:
|
||||
- ./html:/var/www/html
|
||||
- ./config:/etc/nginx/conf.d
|
||||
networks:
|
||||
- webnet
|
||||
```
|
||||
|
||||
```bash
|
||||
# Backup the web service
|
||||
./docker2pbs.py docker-compose.yml web --pbs-repository backup@pbs.local:webapps
|
||||
|
||||
# Creates these archives:
|
||||
# - bind_dot_html.pxar (contains ./html content)
|
||||
# - bind_dot_config.pxar (contains ./config content)
|
||||
```
|
||||
|
||||
### Database Service with Named Volumes
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
database:
|
||||
image: postgres:13
|
||||
volumes:
|
||||
- db_data:/var/lib/postgresql/data
|
||||
- db_backups:/backups
|
||||
environment:
|
||||
POSTGRES_PASSWORD: secret
|
||||
|
||||
volumes:
|
||||
db_data:
|
||||
db_backups:
|
||||
external: true
|
||||
```
|
||||
|
||||
```bash
|
||||
# Backup database service
|
||||
./docker2pbs.py docker-compose.yml database \
|
||||
--pbs-repository dba@pbs.local:databases \
|
||||
--pbs-username backup-user
|
||||
|
||||
# Creates these archives:
|
||||
# - volume_db_data.pxar (PostgreSQL data directory)
|
||||
# - volume_db_backups.pxar (backup directory)
|
||||
```
|
||||
|
||||
## Restore Operations
|
||||
|
||||
See [restore_example.md](restore_example.md) for detailed restore procedures.
|
||||
|
||||
### Quick Restore Example
|
||||
|
||||
```bash
|
||||
# List available backups
|
||||
proxmox-backup-client list --repository user@pbs.example.com:datastore
|
||||
|
||||
# Restore specific volume
|
||||
proxmox-backup-client restore \
|
||||
--repository user@pbs.example.com:datastore \
|
||||
host/web/2024-01-15_10-30-45 \
|
||||
volume_db_data.pxar /var/lib/docker/volumes/db_data/_data
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### PBS Client Setup
|
||||
|
||||
Ensure `proxmox-backup-client` is installed and configured:
|
||||
|
||||
```bash
|
||||
# Install PBS client (Debian/Ubuntu)
|
||||
apt install proxmox-backup-client
|
||||
|
||||
# Configure repository (optional)
|
||||
export PBS_REPOSITORY=user@pbs.example.com:datastore
|
||||
export PBS_PASSWORD=your_password
|
||||
```
|
||||
|
||||
### Supported Volume Types
|
||||
|
||||
- ✅ **Bind Mounts** - Local directories mounted into containers
|
||||
- ✅ **Named Volumes** - Docker-managed volumes
|
||||
- ✅ **External Volumes** - Pre-existing volumes managed outside compose
|
||||
- ❌ **Tmpfs Volumes** - Memory-based volumes (not backed up)
|
||||
- ❌ **Anonymous Volumes** - Unnamed volumes (not backed up)
|
||||
|
||||
### Supported Network Types
|
||||
|
||||
- ✅ **Default Networks** - Docker compose default networks
|
||||
- ✅ **Named Networks** - Custom networks defined in compose
|
||||
- ✅ **External Networks** - Pre-existing networks
|
||||
- ✅ **Bridge Networks** - Standard bridge networks
|
||||
- ✅ **Overlay Networks** - Multi-host networks
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- PBS credentials can be provided via command line or environment variables
|
||||
- Avoid hardcoding passwords in scripts
|
||||
- Use PBS fingerprint verification for secure connections
|
||||
- Consider using PBS tokens instead of passwords
|
||||
- Ensure backup user has minimal required permissions
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Service fails to stop**
|
||||
```bash
|
||||
# Check if service is running
|
||||
docker-compose ps
|
||||
|
||||
# Manually stop if needed
|
||||
docker-compose stop service_name
|
||||
```
|
||||
|
||||
2. **Volume path not found**
|
||||
```bash
|
||||
# Check volume exists
|
||||
docker volume ls
|
||||
docker volume inspect volume_name
|
||||
|
||||
# For bind mounts, verify path exists
|
||||
ls -la /path/to/bind/mount
|
||||
```
|
||||
|
||||
3. **PBS connection fails**
|
||||
```bash
|
||||
# Test PBS connectivity
|
||||
proxmox-backup-client list --repository user@pbs.example.com:datastore
|
||||
|
||||
# Verify credentials
|
||||
echo $PBS_PASSWORD
|
||||
```
|
||||
|
||||
4. **Permission issues**
|
||||
```bash
|
||||
# Ensure script is executable
|
||||
chmod +x docker2pbs.py
|
||||
|
||||
# Check Docker permissions
|
||||
docker ps
|
||||
```
|
||||
|
||||
### Debug Mode
|
||||
|
||||
For verbose output, modify the script to enable debug logging or run with:
|
||||
|
||||
```bash
|
||||
# Show all executed commands
|
||||
bash -x docker2pbs.py docker-compose.yml service --pbs-repository repo --dry-run
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create feature branch (`git checkout -b feature/amazing-feature`)
|
||||
3. Commit changes (`git commit -m 'Add amazing feature'`)
|
||||
4. Push to branch (`git push origin feature/amazing-feature`)
|
||||
5. Open Pull Request
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License - see the LICENSE file for details.
|
||||
|
||||
## Support
|
||||
|
||||
- Create issues for bugs or feature requests
|
||||
- Check existing documentation in `/docs` directory
|
||||
- Review example configurations in `/examples` directory
|
||||
|
||||
## Related Projects
|
||||
|
||||
- [Proxmox Backup Server](https://www.proxmox.com/en/proxmox-backup-server)
|
||||
- [Docker Compose](https://docs.docker.com/compose/)
|
||||
- [proxmox-backup-client](https://pbs.proxmox.com/docs/backup-client.html)
|
||||
Reference in New Issue
Block a user