Netbox: Replication/Migration

A bash script to automatically copy and import database on a different Netbox server.
Intro
This is a simple script that was created to be run on a secondary Netbox Ubuntu server. The purpose is to serve as a backup Netbox server incase primary Netbox server cannot be reached. In my environment Netbox data doesn’t change very often so a non-realtime solution which runs once a day or even once a week is sufficient.
Bonus: This script and its process can also be used to migrate to new Netbox server.
Prerequisites
The script is fairly basic, there are a couple of things to lookout for during the import process. It has the comments and echo commands included to show what is happening at each step.
Both your primary and secondary server will need to have similar setup.
- Same version of Netbox
- Match OS and PostgreSQL user and password (you can modify script to change this)
- When setting up configuration.py on secondary server, change ALLOWED_HOSTS value to match that of secondary server, dont match it with primary server unless you have DNS failover implemented as well.
- Match SECRET_KEY value under configuration.py
To keep things simple, this script doesnt implement LDAP and image attachments replication. You could modify the script easily to do so, I’ve kept mine barebones and simple.
You’ll need to automate/schedule this script to run on the secondary server. Netbox databases are fairly small so the whole process doesnt take very long to run.
SSH setup
As the script is executed from secondary server, it wil need to connect to primary server to run a few things. Therefore, ensure the SSH public key of secondary server is authorized on primary server so there are no password prompts.
Here are the steps on how to accomplish this if you haven’t already done so. These steps are to be followed on secondary server.
- Generate SSH key on secondary server with default settings:
ssh-keygen -t rsa -b 4096
- Copy SSH public key to primary server’s authorized key file:
cat ~/.ssh/id_rsa.pub | ssh [email protected] "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
- SSH into primary server from secondary to test it, there should be no password prompt:
Script
The following need to be modified for your environment before running the script.
PG_DB
: The name of the PostgreSQL databasePG_USER
andPG_PASS
: credentials for PostgreSQL user- Web login info:
SUPERUSER_NAME
,SUPERUSER_EMAIL
,SUPERUSER_PASSWORD
,SUPERUSER_API_TOKEN
PRIMARY_SERVER
: FQDN/IP of primary serverPRIMARY_SERVER_USER
: SSH user on primary server
Expand code below
#!/bin/bash
echo "Creating .pgpass hidden file for PostgreSQL authentication"
# Configure database credentials
PG_HOST="localhost"
PG_PORT="5432"
PG_DB="netbox"
PG_USER="netbox"
PG_PASS="YourPASSWORDhere"
PGPASS_FILE="$HOME/.pgpass"
# Configure web admin credentials
SUPERUSER_NAME="admin"
SUPERUSER_EMAIL="[email protected]"
SUPERUSER_PASSWORD="YourPASSWORDhere"
SUPERUSER_API_TOKEN="YourOwn40CharacterTokenHereKEEPItASecret"
# Write .pgpass for non-interactive access and set permissions
echo "*:$PG_PORT:*:$PG_USER:$PG_PASS" > "$PGPASS_FILE"
chmod 0600 "$PGPASS_FILE"
# Remote server details
PRIMARY_SERVER="PrimaryNetboxServer.example.com"
PRIMARY_SERVER_USER="sshuser01"
echo "Running pg_dump on PrimaryServer via SSH"
ssh "$PRIMARY_SERVER_USER@$PRIMARY_SERVER" "echo '*:$PG_PORT:*:$PG_USER:$PG_PASS' > ~/.pgpass && chmod 600 ~/.pgpass && pg_dump --username=$PG_USER --host=localhost $PG_DB > /tmp/netbox.sql"
if [ $? -ne 0 ]; then
echo "pg_dump failed on PrimaryServer"
exit 1
fi
echo "Copying netbox.sql from PrimaryServer to SecondaryServer"
scp "$PRIMARY_SERVER_USER@$PRIMARY_SERVER:/tmp/netbox.sql" /tmp/netbox.sql
if [ $? -ne 0 ]; then
echo "Failed to copy netbox.sql to SecondaryServer"
exit 1
fi
echo "netbox.sql successfully copied to SecondaryServer"
echo "Stopping netbox services on SecondaryServer"
sudo systemctl stop netbox netbox-rq
echo "Dropping and recreating database"
# Use postgres user
sudo -u postgres psql -c "DROP DATABASE IF EXISTS $PG_DB"
sudo -u postgres psql -c "CREATE DATABASE $PG_DB"
echo "Importing netbox.sql into the new database"
sudo -u postgres psql "$PG_DB" < /tmp/netbox.sql
echo "Granting schema permissions to netbox user"
sudo -u postgres psql -d "$PG_DB" -c "GRANT CREATE ON SCHEMA public TO $PG_USER"
# Wipe all logs from django_admin_log before recreating user, otherwise, it will throw errors.
sudo -u postgres psql netbox -c "DELETE FROM django_admin_log;"
# Activate the virtual environment and run the shell
# Delete 'admin' user if it exists, then create it. Without this, web authentication will fail after restoring database.
source /opt/netbox/venv/bin/activate && \
cd /opt/netbox/netbox && \
./manage.py shell --interface python <<END
from users.models import Token, User
existing = User.objects.filter(username="${SUPERUSER_NAME}").first()
if existing:
print("Deleting existing superuser '${SUPERUSER_NAME}'...")
Token.objects.filter(user=existing).delete()
existing.delete()
print("Creating new superuser '${SUPERUSER_NAME}'...")
u = User.objects.create_superuser('${SUPERUSER_NAME}', '${SUPERUSER_EMAIL}', '${SUPERUSER_PASSWORD}')
Token.objects.create(user=u, key='${SUPERUSER_API_TOKEN}')
print("Superuser '${SUPERUSER_NAME}' created successfully with API token.")
END
echo "Starting netbox services on SecondaryServer"
sudo systemctl start netbox netbox-rq
echo "Finished!"