Client Guide

Complete guide to using the PutPlace client (pp_client.py).

Overview

The PutPlace client is a command-line tool that scans directories, calculates file metadata, and uploads it to a PutPlace server. It features:

📁 Recursive directory scanning
🔐 Username/password authentication with JWT tokens
🎯 Pattern-based file exclusion
🚀 Automatic file deduplication
🎨 Rich console output
⚙️ Flexible configuration (CLI/env/file)

Installation

The client is included in the PutPlace repository:

# Clone repository
git clone https://github.com/jdrumgoole/putplace.git
cd putplace

# Install dependencies
pip install -e .

# Or install required packages directly
pip install httpx rich configargparse

Quick Start

Create User Account

You need a user account to use the client. Ask your server administrator to create one, or if you’re the admin, users are created via the server’s registration endpoint.

The admin user is automatically created on first server startup (check server logs for credentials).

First Scan

# Set your credentials
export PUTPLACE_USERNAME="your-username"
export PUTPLACE_PASSWORD="your-password"

# Test scan (dry run)
python pp_client.py /tmp --dry-run

# Real scan
python pp_client.py /tmp

Usage

Basic Syntax

python pp_client.py [OPTIONS] PATH

Options

Required Options

PATH (positional)

Path to scan (file or directory)
Supports absolute and relative paths

python pp_client.py /var/www
python pp_client.py ~/Documents
python pp_client.py .

--username USERNAME or -u USERNAME

Username for authentication
Can also use PUTPLACE_USERNAME environment variable
Can also set in config file

python pp_client.py /tmp --username "admin"
python pp_client.py /tmp -u "admin"

--password PASSWORD or -p PASSWORD

Password for authentication
Can also use PUTPLACE_PASSWORD environment variable
Can also set in config file

python pp_client.py /tmp --password "your-password"
python pp_client.py /tmp -p "your-password"

Optional Options

--url URL

PutPlace server API endpoint
Default: http://localhost:8000/put_file

python pp_client.py /tmp --url "https://putplace.example.com/put_file"

--hostname HOSTNAME

Override auto-detected hostname
Useful for custom naming

python pp_client.py /tmp --hostname "web-server-01"

--ip IP

Override auto-detected IP address
Useful when multiple IPs exist

python pp_client.py /tmp --ip "192.168.1.100"

--exclude PATTERN

Exclude files matching pattern
Can be used multiple times
Supports wildcards

python pp_client.py /tmp --exclude "*.log" --exclude ".git"

--dry-run

Scan files but don’t send to server
Useful for testing

python pp_client.py /tmp --dry-run

--verbose or -v

Enable verbose output
Shows detailed progress

python pp_client.py /tmp --verbose
python pp_client.py /tmp -v

--config PATH

Path to configuration file
Default: pp_client.conf or ~/pp_client.conf

python pp_client.py /tmp --config ~/my-config.conf

--help or -h

Show help message

python pp_client.py --help

Configuration

Configuration Priority

Settings are applied in this order (highest to lowest priority):

Command-line arguments (highest)
Environment variables
Configuration file (lowest)

Example:

# Config file has: username = file-user, password = file-pass
# Environment has: PUTPLACE_USERNAME=env-user, PUTPLACE_PASSWORD=env-pass
# Command line has: --username cli-user --password cli-pass

# Result: cli-user and cli-pass are used

Configuration File

Create a configuration file to avoid repeating options:

~/pp_client.conf:

[DEFAULT]
url = http://localhost:8000/put_file
username = your-username
password = your-password
exclude = .git
exclude = __pycache__
exclude = *.log

Set secure permissions:

chmod 600 ~/pp_client.conf

Use:

# All settings from config file
python pp_client.py /var/www

See Configuration Reference for all options.

Authentication

Three Methods

1. Command Line (Quick Testing)

python pp_client.py /tmp --username "admin" --password "your-password"

Pros: Quick for testing Cons: Visible in shell history and process list

2. Environment Variable (Recommended for Scripts)

export PUTPLACE_USERNAME="admin"
export PUTPLACE_PASSWORD="your-password"
python pp_client.py /tmp

Pros: Not in command history, works across commands Cons: Visible in environment

Make persistent:

# For bash
echo 'export PUTPLACE_USERNAME="admin"' >> ~/.bashrc
echo 'export PUTPLACE_PASSWORD="your-password"' >> ~/.bashrc
source ~/.bashrc

# For zsh
echo 'export PUTPLACE_USERNAME="admin"' >> ~/.zshrc
echo 'export PUTPLACE_PASSWORD="your-password"' >> ~/.zshrc
source ~/.zshrc

3. Config File (Recommended for Production)

# Create config
cat > ~/pp_client.conf << 'EOF'
[DEFAULT]
username = admin
password = your-password
EOF

chmod 600 ~/pp_client.conf

# Use
python pp_client.py /tmp

Pros: Most secure, persistent Cons: Requires file setup

Security Best Practices

DO:

Use separate user accounts per client/server
Set file permissions to 600 on config files
Use strong passwords
Change passwords regularly
Use environment variables or config files in production

DON’T:

Commit passwords to version control
Share passwords between users
Use command-line passwords in production
Use overly permissive file permissions

File Exclusion

Exclude Patterns

Use --exclude to skip files:

File extensions:

--exclude "*.log"      # All .log files
--exclude "*.tmp"      # All .tmp files
--exclude "*.pyc"      # Compiled Python files

Directories:

--exclude ".git"           # Git repositories
--exclude "node_modules"   # Node.js dependencies
--exclude "__pycache__"    # Python cache
--exclude ".venv"          # Virtual environments

Wildcards:

--exclude "test_*"     # Files starting with test_
--exclude "*~"         # Backup files
--exclude ".*.swp"     # Vim swap files

Common Exclude Patterns

Python projects:

python pp_client.py /project \
  --exclude ".git" \
  --exclude "__pycache__" \
  --exclude "*.pyc" \
  --exclude ".venv" \
  --exclude "venv" \
  --exclude ".pytest_cache" \
  --exclude "*.egg-info"

Node.js projects:

python pp_client.py /project \
  --exclude ".git" \
  --exclude "node_modules" \
  --exclude "dist" \
  --exclude "build" \
  --exclude "*.log"

Web servers:

python pp_client.py /var/www \
  --exclude ".git" \
  --exclude "*.log" \
  --exclude "cache" \
  --exclude "tmp" \
  --exclude ".DS_Store"

Config File Exclusions

Put common exclusions in config file:

[DEFAULT]
# Universal exclusions
exclude = .git
exclude = .DS_Store
exclude = Thumbs.db

# Python
exclude = __pycache__
exclude = *.pyc
exclude = .venv
exclude = venv

# Node.js
exclude = node_modules

# Logs and temporary files
exclude = *.log
exclude = *.tmp
exclude = tmp
exclude = cache

Examples

Example 1: Simple Scan

Scan a directory with default settings:

export PUTPLACE_USERNAME="admin"
export PUTPLACE_PASSWORD="your-password"
python pp_client.py /var/www

Output:

PutPlace Client
  Path: /var/www
  Hostname: web-server-01
  IP: 192.168.1.100
  URL: http://localhost:8000/put_file
  Username: admin

Found 150 files to process
Processing files... ━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:05
✓ Processing complete

Summary:
  Total files: 150
  Successful: 148
  Failed: 2
  Files uploaded: 35

Example 2: Dry Run

Test without sending data:

python pp_client.py /tmp --dry-run

Output shows what would be sent but doesn’t actually send.

Example 3: Remote Server

Scan and send to remote server:

python pp_client.py /var/www \
  --url "https://putplace.example.com/put_file" \
  --username "admin" \
  --password "production-password"

Example 4: Custom Hostname

Override auto-detected hostname:

python pp_client.py /var/www \
  --hostname "web-prod-01" \
  --ip "10.0.1.50"

Example 5: Multiple Exclusions

Exclude multiple patterns:

python pp_client.py /home/user \
  --exclude ".git" \
  --exclude "node_modules" \
  --exclude "__pycache__" \
  --exclude "*.log" \
  --exclude "*.tmp" \
  --exclude ".venv"

Example 6: Verbose Output

See detailed progress:

python pp_client.py /tmp --verbose

Output:

PutPlace Client
  Path: /tmp
  ...

Scanning directory: /tmp
Found file: /tmp/file1.txt (1234 bytes)
Found file: /tmp/file2.txt (5678 bytes)
...

Processing: /tmp/file1.txt
  SHA256: abc123...
  Sending metadata... ✓
  Upload required: No (file already exists)

Processing: /tmp/file2.txt
  SHA256: def456...
  Sending metadata... ✓
  Upload required: Yes
  Uploading content... ✓

Workflows

Development Workflow

# 1. Get credentials from server admin
export PUTPLACE_USERNAME="dev-user"
export PUTPLACE_PASSWORD="dev-password"

# 2. Test connection with dry run
python pp_client.py /tmp --dry-run

# 3. Scan development directory
python pp_client.py ~/projects/myapp \
  --exclude ".git" \
  --exclude "node_modules" \
  --exclude ".venv"

Production Workflow

# 1. Create config file
cat > ~/pp_client.conf << 'EOF'
[DEFAULT]
url = https://putplace.example.com/put_file
username = prod-user
password = production-password
exclude = .git
exclude = *.log
exclude = tmp
EOF

chmod 600 ~/pp_client.conf

# 2. Test with dry run
python pp_client.py /var/www --dry-run

# 3. Run actual scan
python pp_client.py /var/www

# 4. Set up cron job for daily scans
echo "0 2 * * * /usr/bin/python3 /path/to/pp_client.py /var/www" | crontab -

Multi-Environment Workflow

# Development config
cat > ~/pp_client.conf.dev << 'EOF'
url = http://dev-putplace:8000/put_file
username = dev-user
password = dev-password
EOF

# Staging config
cat > ~/pp_client.conf.staging << 'EOF'
url = https://staging-putplace.example.com/put_file
username = staging-user
password = staging-password
EOF

# Production config
cat > ~/pp_client.conf.prod << 'EOF'
url = https://putplace.example.com/put_file
username = prod-user
password = prod-password
EOF

# Use with --config flag
python pp_client.py /var/www --config ~/pp_client.conf.dev
python pp_client.py /var/www --config ~/pp_client.conf.staging
python pp_client.py /var/www --config ~/pp_client.conf.prod

Automated Scanning

Cron Jobs

Daily Scan at 2 AM

# Edit crontab
crontab -e

# Add line
0 2 * * * /usr/bin/python3 /path/to/pp_client.py /var/www

Hourly Scan

0 * * * * /usr/bin/python3 /path/to/pp_client.py /var/www

Weekly Scan (Sundays at 3 AM)

0 3 * * 0 /usr/bin/python3 /path/to/pp_client.py /var/www

With Logging

0 2 * * * /usr/bin/python3 /path/to/pp_client.py /var/www >> /var/log/pp_client.log 2>&1

systemd Timer

Create service: /etc/systemd/system/pp_client.service

[Unit]
Description=PutPlace Client Scan
After=network.target

[Service]
Type=oneshot
User=www-data
Group=www-data
Environment="PUTPLACE_USERNAME=prod-user"
Environment="PUTPLACE_PASSWORD=prod-password"
ExecStart=/usr/bin/python3 /path/to/pp_client.py /var/www
StandardOutput=journal
StandardError=journal

Create timer: /etc/systemd/system/pp_client.timer

[Unit]
Description=PutPlace Client Daily Scan
Requires=pp_client.service

[Timer]
OnCalendar=daily
OnCalendar=02:00
Persistent=true

[Install]
WantedBy=timers.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable pp_client.timer
sudo systemctl start pp_client.timer

# Check status
sudo systemctl status pp_client.timer
sudo systemctl list-timers

Output and Progress

Console Output

The client provides rich console output with:

✅ Configuration summary
📊 Progress bar during scanning
📈 Real-time counters
✓ Success/failure indicators
📋 Final summary

Exit Codes

 # Success - all files processed
 # Partial failure - some files failed
 # Complete failure - no files processed

Use in scripts:

python pp_client.py /var/www
if [ $? -eq 0 ]; then
    echo "All files processed successfully"
elif [ $? -eq 1 ]; then
    echo "Some files failed"
else
    echo "Complete failure"
fi

Troubleshooting

Authentication Issues

Problem: “Both username and password are required”

✗ Both username and password are required for authentication

Solution: Provide both credentials via:

--username and --password flags
PUTPLACE_USERNAME and PUTPLACE_PASSWORD environment variables
username and password in config file

Problem: “Login failed: 401”

✗ Login failed: 401
  Incorrect username or password

Causes:

Invalid username or password
User account disabled
Wrong server URL

Solution:

# Test credentials
curl -X POST http://localhost:8000/api/login \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "your-password"}'

# If you forgot password, contact server admin to reset

Connection Issues

Problem: “Connection refused”

Failed to send file.txt: Cannot connect to host localhost:8000

Causes:

Server not running
Wrong URL
Firewall blocking

Solution:

# Check server is running
curl http://localhost:8000/health

# Check URL
python pp_client.py /tmp --url "http://correct-server:8000/put_file"

# Check firewall
telnet localhost 8000

File Access Issues

Problem: “Permission denied”

Error scanning /var/www/private: Permission denied

Solution:

# Run with appropriate user
sudo -u www-data python pp_client.py /var/www

# Or fix permissions
sudo chmod -R +r /var/www

Large Directory Scans

Problem: Scan is slow

Solutions:

Use exclusions to skip unnecessary files
Scan subdirectories separately
Use --verbose to monitor progress

# Exclude large directories
python pp_client.py /var/www \
  --exclude "cache" \
  --exclude "tmp" \
  --exclude "*.log"

# Scan subdirectories separately
python pp_client.py /var/www/app
python pp_client.py /var/www/static

Advanced Usage

Custom Scripts

Integrate client into your scripts:

#!/bin/bash
# backup-and-scan.sh

# Backup directories
DIRS="/var/www /etc /opt/myapp"

for dir in $DIRS; do
    echo "Scanning $dir..."
    python pp_client.py "$dir" \
        --url "https://putplace.example.com/put_file" \
        --username "$PUTPLACE_USERNAME" \
        --password "$PUTPLACE_PASSWORD" \
        --exclude ".git" \
        --exclude "*.log"

    if [ $? -ne 0 ]; then
        echo "ERROR: Failed to scan $dir"
        exit 1
    fi
done

echo "All directories scanned successfully"

Monitoring Integration

Send results to monitoring system:

#!/bin/bash
# scan-with-monitoring.sh

OUTPUT=$(python pp_client.py /var/www 2>&1)
EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
    # Send success metric to monitoring
    curl -X POST https://monitoring.example.com/metric \
        -d "service=pp_client&status=success"
else
    # Send failure metric and alert
    curl -X POST https://monitoring.example.com/metric \
        -d "service=pp_client&status=failure"

    # Send alert
    echo "$OUTPUT" | mail -s "PutPlace scan failed" ops@example.com
fi

Parallel Scanning

Scan multiple directories in parallel:

#!/bin/bash
# parallel-scan.sh

python pp_client.py /var/www &
python pp_client.py /etc &
python pp_client.py /opt &

wait

echo "All scans complete"

Performance Tips

Use exclusions - Skip unnecessary files

--exclude "*.log" --exclude "cache" --exclude "tmp"

Scan subdirectories separately - For very large directories

for dir in /var/www/*; do
    python pp_client.py "$dir"
done

Use dry run for testing - Verify setup without sending
```
python pp_client.py /var/www --dry-run
```

Run during off-peak hours - Use cron for scheduled scans

0 2 * * * /usr/bin/python3 /path/to/pp_client.py /var/www

Monitor progress - Use verbose mode for long scans

python pp_client.py /large/directory --verbose

Graceful Interrupt Handling

The PutPlace client handles Ctrl-C (SIGINT) gracefully, allowing you to stop long-running scans safely.

How It Works

First Ctrl-C:

Finishes processing the current file
Exits cleanly after current operation
Shows partial completion status
Returns exit code 1 (indicating incomplete)

Second Ctrl-C:

Forces immediate termination
Standard Python KeyboardInterrupt behavior

Example Usage

# Start scanning a large directory
pp_client --path /large/directory

# Press Ctrl-C once to stop gracefully
# (Current file completes, then exits)

# Output:
# ⚠ Interrupt received, finishing current file and exiting...
# (Press Ctrl-C again to force quit)
#
# Processing interrupted by user
#
# Results:
#   Status: Interrupted (partial completion)
#   Total files: 1000
#   Successful: 247
#   Failed: 0
#   Remaining: 753

Use Cases

1. Testing:

# Start scan to verify configuration
pp_client --path /large/directory

# Once you see it's working, press Ctrl-C to stop

2. Resource Management:

# Stop scan if system load is too high
pp_client --path /data
# ... system load warning appears ...
# Press Ctrl-C to stop gracefully

3. Time-Limited Scans:

# Run scan for a few minutes to sample data
pp_client --path /var/www
# Press Ctrl-C when you have enough samples

4. Scripting with Timeout:

#!/bin/bash
# Start scan in background
pp_client --path /data &
PID=$!

# Wait for 5 minutes
sleep 300

# Gracefully interrupt if still running
if kill -0 $PID 2>/dev/null; then
    kill -INT $PID  # Send SIGINT (same as Ctrl-C)
    wait $PID
fi

Exit Codes

When interrupted:

Exit code 1: Indicates partial completion
Same as when some files fail processing
Useful for automation/monitoring

pp_client --path /data
EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
    echo "Complete success"
elif [ $EXIT_CODE -eq 1 ]; then
    echo "Partial completion or interrupted"
    # Could be interrupted or some failures
fi

Best Practices

Always let the current file finish - First Ctrl-C ensures data consistency
Check the summary - Review how many files were processed before interrupt

Resume where you left off - Use exclusions to skip already-processed files:

# First scan (interrupted after 100 files)
pp_client --path /data

# Resume by scanning remaining directories
pp_client --path /data/subdirectory

Monitor long scans - Use --verbose to see progress and know when to interrupt:
```
pp_client --path /large/directory --verbose
```

Next Steps

Authentication Guide - User authentication and JWT tokens
Configuration Reference - All configuration options
Troubleshooting - Common issues and solutions
API Reference - REST API documentation