Skip to main content

setup nebula mesh

Source

Tags:

I've been using Wireguard for years to access some key servers from wherever I am without even thinking about it, recently I've been testing a few new services and decided to try another method for getting access to them, one that would not require any action on existing enabled devices. I remember trying nebula some years back, I think I didn't bother making it permentant as Wireguard was working fine for me at the time.
To set this up the plan is to:
  • Setup nebula-cert locally for genertation of certs.
  • Manually setup 1 x standard node.
  • Manually setup 1 x lighthouse node (lighthouse is a publically accessible server, at least one is required for best results).
  • Automate the process to add additional standard nodes using python or ansible or a combination of both.

Step 1: Setup of certs and initial node

Setup instuctions and downloads available from the Nebula Github page. After downloading the binaries we generate two certs initially for the lighthouse and first node.

  • To keep the certs secure I copied nebula-cert to a .nebula folder and updated permissions to 700. All certs will be generated in this folder and copied to where they're needed.
  • Create our CA cert in .nebula using ./nebula-cert ca -name "Org Name"
  • Next we generate certs in that same folder for our initial two nodes, choosing our preferred subnet:

  • ./nebula-cert sign -name "light.domain.com" -ip "192.168.100.1/24"

  • ./nebula-cert sign -name "desktop-pc1" -ip "192.168.100.2/24"

  • Note there is an option to include a -groups flag when generating the cert if you wish to segment some of your traffic but I don't need this.

Step 2: Setup the nodes

There's a default config.yml file available on the Nebula Github page. We download that and make the required changes for the specific nodes. The config file is really well documented, for my setup there's not much config required, the main changes are:

  • path to the cert files
  • On the lighthouse node, you'll need to ensure am_lighthouse: true is set.
  • On the individual hosts, ensure the lighthouse is defined properly in the static_host_map section, and is added to the lighthouse -> hosts section.
  • By default only icmp and 443 traffic is allowed on the nebula network, add anything else you need, e.g. to allow ssh I added this to the bottom of the config file:
    # Allow ssh
    - port: 22
      proto: tcp
      host: any

On each node we copy the files below to a directory of our choice, in my case I copied to /usr/local/bin/nebula

  • ca.crt
  • desktop-pc1.crt
  • desktop-pc1.key
  • nebula (binary)
  • desktop-pc1.yml

Rather than running manually we're going to set nebula up as a systemd service, to do this we create a service file /etc/systemd/system/nebula.service with the syntax below and repeat on our two initial nodes.:

[Unit]
Description=Nebula Mesh Service
After=network.target

[Service]
ExecStart=/usr/local/bin/nebula/nebula -config /usr/local/bin/nebula/desktop-pc1.yml
Restart=always

[Install]
WantedBy=multi-user.target

Finally we can start the service with systemctl service start nebula && systemctl service enable nebula

/.nebula$ sudo systemctl start nebula
/.nebula$ sudo systemctl status nebula

  nebula.service - Nebula Mesh Service
     Loaded: loaded (/etc/systemd/system/nebula.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-07-20 23:02:20 IST; 11s ago
   Main PID: 47341 (nebula)
  • note, if you just want to test nebula you can manually run ./nebula -config /path/to/config.yml
  • note, if your service won't start, nebula provides very useful info in the logs so just follow journalctl while starting the service to see whats wrong :)
  • note, don't foget to create an allow 4242 udp rule on your firewall.

Step 3: Create our Nebula config on each server.

Finally to simplify setting up nebula on future nodes I created an Ansible playbook, there's one templates that I can't include (the config file) but really only the file paths are "templateified" so it should be straight forward to setup. The playbook is available here

That's all there was to it, I'm planning to setup a self hosted wallabag instance and will be enabling nebula on that to allow access from any of my devices without having to expose the service, I will update here if that presents any nebula gotchas.