From baa8f499a12d8deccb1b91bf462c9074aa042849 Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Wed, 22 Apr 2020 19:38:44 +0000 Subject: [PATCH 1/9] Resolve "Support Ubuntu 20.04 LTS" --- README.md | 34 ++++++++++- scripts/subinstallers/docker.sh | 79 ++++---------------------- scripts/subinstallers/docker_manual.sh | 71 +++++++++++++++++++++++ scripts/subinstallers/platform.sh | 3 + scripts/subinstallers/wireguard.sh | 38 ++++++++----- 5 files changed, 142 insertions(+), 83 deletions(-) create mode 100755 scripts/subinstallers/docker_manual.sh diff --git a/README.md b/README.md index 2b4f5de..e780665 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,38 @@ Within Mistborn is a panel to enable and manage these free extra services (off b - [Tor](https://www.torproject.org): The Onion Router. One tool in the arsenal of online security and privacy. - [Jitsi](https://jitsi.org): Multi-platform open-source video conferencing +# Quickstart +Tested Operating Systems (in order of thoroughness): +- Ubuntu 18.04 LTS +- Ubuntu 20.04 LTS +- Debian 10 (Buster) +- Raspbian Buster + +Recommended System Specifications: + +| Use Case | Description | RAM | Hard Disk | +|------------------------|-------------------------------------------------------------------------------|-------|-----------| +| Bare bones | Wireguard, Pihole (no Cockpit, no extra services) | 1 GB | 10 GB | +| Default | Bare bones + Cockpit | 2 GB | 10 GB | +| Low-resource services | Default + Bitwarden, Tor, Syncthing | 3 GB | 15 GB | +| High-resource services | Default + Jitsi, Nextcloud, Jellyfin, Rocket.Chat, Home Assistant, OnlyOffice | 4 GB+ | 25 GB+ | + +Starting from base installation +``` +git clone https://gitlab.com/cyber5k/mistborn.git +sudo bash ./mistborn/scripts/install.sh +``` + +Get default admin Wireguard profile +*wait 1 minute after "Mistborn Installed" message* +``` +sudo docker-compose -f /opt/mistborn/base.yml run --rm django python manage.py getconf admin default +``` + +Connect via Wireguard then visit `http://home.mistborn` + +For more information, see the `Installation` section below. + # Network Diagram ![Mistborn Network Diagram](https://gitlab.com/cyber5k/public/-/raw/master/graphics/mistborn_network.png) @@ -54,7 +86,7 @@ In Mistborn, Gateways are upstream from the VPN server so connections to third-p The Gateway adds an extra network hop. DNS is still resolved in Mistborn so pihole is still blocking ads. # Installation -Mistborn is regularly tested on Ubuntu 18.04 LTS (DigitalOcean droplet with 2 GB RAM). It has also been successfully used on Debian Buster and Raspbian Buster systems (though not regularly tested). +Mistborn is regularly tested on Ubuntu 18.04 LTS (DigitalOcean droplet with 2 GB RAM). It has also been successfully used on Debian Buster and Raspbian Buster systems (though not regularly tested). Additionally tested on Ubuntu 20.04 LTS. Clone the git repository and run the install script: ``` diff --git a/scripts/subinstallers/docker.sh b/scripts/subinstallers/docker.sh index b962864..1542207 100755 --- a/scripts/subinstallers/docker.sh +++ b/scripts/subinstallers/docker.sh @@ -1,72 +1,17 @@ #!/bin/bash -# Docker -figlet "Mistborn: Installing Docker" - -# dependencies -echo "Installing Docker dependencies" -sudo apt-get install -y \ - apt-transport-https \ - ca-certificates \ - curl \ - gnupg-agent \ - software-properties-common - -# Docker repo key -echo "Adding docker repository key" -if [ "$DISTRO" == "ubuntu" ]; then - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - -elif [ "$DISTRO" == "debian" ]; then - curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add - -elif [ "$DISTRO" == "raspbian" ]; then - curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | sudo apt-key add - -fi - -# Docker repo to source list -echo "Adding docker to sources list" -if [ "$DISTRO" == "ubuntu" ]; then - sudo add-apt-repository -y \ - "deb https://download.docker.com/linux/ubuntu \ - $(lsb_release -cs) \ - stable" -elif [ "$DISTRO" == "debian" ]; then - sudo add-apt-repository -y \ - "deb https://download.docker.com/linux/debian \ - $(lsb_release -cs) \ - stable" -elif [ "$DISTRO" == "raspbian" ]; then - echo "deb [arch=armhf] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \ - $(lsb_release -cs) stable" | \ - sudo tee /etc/apt/sources.list.d/docker.list +sudo apt update +sudo apt install -y python python3-pip python3-setuptools libffi-dev python3-dev libssl-dev + +if [ "$DISTRO" == "ubuntu" ] && [ "$VERSION_ID" == "20.04" ]; then + echo "Automated Docker install" + sudo apt-get install -y docker-compose +else + echo "Manual Docker installation" + source ./scripts/subinstallers/docker_manual.sh fi -# install Docker -echo "Installing docker" -sudo apt-get update - -if [ "$DISTRO" == "ubuntu" ] || [ "$DISTRO" == "debian" ]; then - sudo apt-get install -y docker-ce docker-ce-cli containerd.io -elif [ "$DISTRO" == "raspbian" ]; then - sudo apt install -y --no-install-recommends \ - docker-ce \ - cgroupfs-mount +# set docker-compose path used in Mistborn +if [ ! -f /usr/local/bin/docker-compose ]; then + sudo ln -s $(which docker-compose) /usr/local/bin/docker-compose fi - -# Docker group -sudo usermod -aG docker $USER - -# Docker Compose -echo "Installing Docker Compose" -#if [ "$DISTRO" == "ubuntu" ] || [ "$DISTRO" == "debian" ]; then -# sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose -# sudo chmod +x /usr/local/bin/docker-compose -#elif [ "$DISTRO" == "raspbian" ]; then -# Install required packages -sudo apt update -sudo apt install -y python python3-pip python3-setuptools libffi-dev python-backports.ssl-match-hostname python3-dev libssl-dev - -# Install Docker Compose from pip -# This might take a while -sudo pip3 install docker-compose -#fi - diff --git a/scripts/subinstallers/docker_manual.sh b/scripts/subinstallers/docker_manual.sh new file mode 100755 index 0000000..176aba9 --- /dev/null +++ b/scripts/subinstallers/docker_manual.sh @@ -0,0 +1,71 @@ +#!/bin/bash + +# Docker +figlet "Mistborn: Installing Docker" + +# dependencies +echo "Installing Docker dependencies" +sudo apt-get install -y \ + apt-transport-https \ + ca-certificates \ + curl \ + gnupg-agent \ + software-properties-common + +# Docker repo key +echo "Adding docker repository key" +if [ "$DISTRO" == "ubuntu" ]; then + curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - +elif [ "$DISTRO" == "debian" ]; then + curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add - +elif [ "$DISTRO" == "raspbian" ]; then + curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | sudo apt-key add - +fi + +# Docker repo to source list +echo "Adding docker to sources list" +if [ "$DISTRO" == "ubuntu" ]; then + sudo add-apt-repository -y \ + "deb https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) \ + stable" +elif [ "$DISTRO" == "debian" ]; then + sudo add-apt-repository -y \ + "deb https://download.docker.com/linux/debian \ + $(lsb_release -cs) \ + stable" +elif [ "$DISTRO" == "raspbian" ]; then + echo "deb [arch=armhf] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \ + $(lsb_release -cs) stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list +fi + +# install Docker +echo "Installing docker" +sudo apt-get update + +if [ "$DISTRO" == "ubuntu" ] || [ "$DISTRO" == "debian" ]; then + sudo apt-get install -y docker-ce docker-ce-cli containerd.io +elif [ "$DISTRO" == "raspbian" ]; then + sudo apt install -y --no-install-recommends \ + docker-ce \ + cgroupfs-mount +fi + +# Docker group +sudo usermod -aG docker $USER + +# Docker Compose +echo "Installing Docker Compose" +#if [ "$DISTRO" == "ubuntu" ] || [ "$DISTRO" == "debian" ]; then +# sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose +# sudo chmod +x /usr/local/bin/docker-compose +#elif [ "$DISTRO" == "raspbian" ]; then +# Install required packages +sudo apt install -y python-backports.ssl-match-hostname + +# Install Docker Compose from pip +# This might take a while +sudo pip3 install docker-compose +#fi + diff --git a/scripts/subinstallers/platform.sh b/scripts/subinstallers/platform.sh index 3c18fce..a619a95 100755 --- a/scripts/subinstallers/platform.sh +++ b/scripts/subinstallers/platform.sh @@ -4,11 +4,14 @@ # Determine OS platform UNAME=$(uname | tr "[:upper:]" "[:lower:]") DISTRO="" +VERSION_ID="" # If Linux, try to determine specific distribution if [ "$UNAME" == "linux" ]; then # use /etc/os-release to get distro DISTRO=$(cat /etc/os-release | awk -F= '/^ID=/{print $2}') + VERSION_ID=$(cat /etc/os-release | awk -F= '/^VERSION_ID=/{print $2}' | tr -d '"') fi figlet "UNAME: $UNAME" figlet "DISTRO: $DISTRO" +figlet "VERSION: $VERSION_ID" diff --git a/scripts/subinstallers/wireguard.sh b/scripts/subinstallers/wireguard.sh index d027f79..93b2490 100755 --- a/scripts/subinstallers/wireguard.sh +++ b/scripts/subinstallers/wireguard.sh @@ -2,23 +2,31 @@ figlet "Mistborn: Installing Wireguard" -# Wireguard -if [ "$DISTRO" == "raspbian" ]; then - echo "Adding Wireguard repo keys" - sudo apt-get install -y dirmngr - sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B48AD6246925553 - sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7638D0442B90D010 - sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC +# if wireguard not in current repositories +if ! $(sudo apt-cache show wireguard > /dev/null 2>&1) ; then + # install PPAs + + echo "Adding Wireguard PPAs" + + # Wireguard + if [ "$DISTRO" == "raspbian" ]; then + echo "Adding Wireguard repo keys" + sudo apt-get install -y dirmngr + sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B48AD6246925553 + sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7638D0442B90D010 + sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC + fi + + if [ "$DISTRO" == "ubuntu" ]; then + # Ubuntu + sudo add-apt-repository -y ppa:wireguard/wireguard + elif [ "$DISTRO" == "debian" ] || [ "$DISTRO" == "raspbian" ]; then + # Debian + sudo bash -c 'echo "deb http://deb.debian.org/debian/ unstable main" > /etc/apt/sources.list.d/unstable.list' + sudo bash -c "printf 'Package: *\nPin: release a=unstable\nPin-Priority: 90\n' > /etc/apt/preferences.d/limit-unstable" + fi fi echo "Installing Wireguard" -if [ "$DISTRO" == "ubuntu" ]; then - # Ubuntu - sudo add-apt-repository -y ppa:wireguard/wireguard -elif [ "$DISTRO" == "debian" ] || [ "$DISTRO" == "raspbian" ]; then - # Debian - sudo bash -c 'echo "deb http://deb.debian.org/debian/ unstable main" > /etc/apt/sources.list.d/unstable.list' - sudo bash -c "printf 'Package: *\nPin: release a=unstable\nPin-Priority: 90\n' > /etc/apt/preferences.d/limit-unstable" -fi sudo apt-get update sudo apt-get install -y openresolv wireguard From 7b3d440eab4f1a108bbbbb17860fbd45325c1272 Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Thu, 23 Apr 2020 21:47:18 +0000 Subject: [PATCH 2/9] Resolve "Feature: Blacklist telemetry IPs (outbound)" --- README.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index e780665..13f3090 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ A platform for easily managing your cloud server and Wireguard access [[_TOC_]] # What is Mistborn -The term [Mistborn](http://www.brandonsanderson.com/the-mistborn-saga-the-original-trilogy) comes from a type of powerful Allomancer in Brandon Sanderson's Cosmere. +The term [Mistborn](http://www.brandonsanderson.com/the-mistborn-saga-the-original-trilogy) is inspired by a type of powerful Allomancer in Brandon Sanderson's Cosmere. Mistborn started as a passion project for a husband and father protecting his family. Certain family members insisted on connecting their devices to free WiFi networks. We needed a way to secure all family devices with a solid VPN (Wireguard). Once we had that we wanted to control DNS to block ads to all devices and block malicious and pornographic websites across all family devices. Then we wanted chat, file-sharing, and webchat services that we could use for ourselves without entrusting our data to some big tech company. And then... home automation. I know I'll be adding services as I go so I made that easy to do. @@ -72,6 +72,13 @@ Mistborn protects your data in a variety of ways: - The Mistborn firewall blocks unsolicited incoming internet packets - Pi-hole running on Mistborn blocks outgoing internet requests to configurable blocked domains (ads, malicious/phishing domains, etc.) +# Coppercloud +Pihole provides a way to block outgoing DNS requests for given lists of blocked domains. Coppercloud provides a way to block outgoing network calls of all types to given lists of IP addresses (IPv4 only for now). This is especially useful for blocking outgoing telemetry (data and state sharing) to owners of software running on all of your devices. + +![Mistborn Coppercloud IP Filtering](https://gitlab.com/cyber5k/public/-/raw/master/graphics/home.mistborn_coppercloud_.png) + +This example shows Coppercloud blocking a list of Microsoft IP addresses on a network with Windows 10 clients. + # Gateways I was getting frustrated at being forced to choose between being connected to my VPN and using streaming services that I have paid for. From f182b12409ddf024583a68a5748f0bb640edf5da Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Fri, 24 Apr 2020 21:21:39 +0000 Subject: [PATCH 3/9] Resolve "README: Add Roadmap and Quickstart" --- README.md | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 13f3090..680d166 100644 --- a/README.md +++ b/README.md @@ -229,7 +229,7 @@ On Gateway: # Troubleshooting -Once you're connected to Wireguard you should see .mistborn domains and the internet should work as expected. Be sure to use http (http://home.mistborn). Wireguard is the encrypted channel so we're not bothering with TLS certs. Here are some things to check if you have issues: +Once you're connected to Wireguard you should see .mistborn domains and the internet should work as expected. Be sure to use http (http://home.mistborn). Wireguard is the encrypted channel so there's usually no need to bother with TLS certs (WebRTC functionality and some mobile apps require TLS so it is available). Here are some things to check if you have issues: See if any docker containers are stopped: ``` @@ -268,6 +268,39 @@ Be sure to restart Docker afterward: sudo systemctl restart docker ``` +# Technical and Security Insights +These are some notes regarding the technical design and implementations of Mistborn. Feel free to contact me for additional details. + +## Attack Surface +- **Wireguard**: Wireguard is the only way in to Mistborn. When new Wireguard profiles are generated they are attached to a random UDP port. Wireguard does not respond to unauthenticated traffic. External probes on the active Wireguard listening ports are not logged and do not appear on the Metrics page. +- **SSH**: If Mistborn is installed over SSH (most common) then an iptables rule is added allowing future SSH connections from the same source IP address. All other external SSH is blocked. Internal SSH (over the Wireguard tunnels) is allowed. +- **Traefik**: Iptables closes web ports (TCP 80 and 443) from external access and additonally all web interfaces are behind the Traefik reverse-proxy. All web requests (e.g. home.mistborn) must be resolved by Mistborn DNS (Pihole/dnsmasq) and originate from a Wireguard tunnel. +- **Docker**: When Docker exposes a port it creates a PREROUTING rule in the NAT table to catch eligible network requests. This means that even if your INPUT chain policy is DROP, your docker containers with exposed ports can receive and respond to traffic. Whenever Mistborn brings up a docker container with an exposed port it creates an iptables rule to block external traffic to that service. + +## Firewall +- **IPtables**: Iptables rules and chains are manipulated directly. If UFW is present it is disabled. IPtables-persistent is used to save a simple set of secure default rules (most importantly setting the INPUT and FORWARD policies to DROP and allowing ESTABLISHED and RELATED traffic) that will be effective immediately upon system startup. Additional rules and chains are created by Docker on startup. Mistborn also creates some iptables chains during installation that are saved in the persistent rules. Mistborn iptables chains and rules are designed to work with Docker's with logic that is easy to follow. A power cycle will always result in a working state. +- **PostUp/PostDown**: Wireguard configuration files on Mistborn include PostUp and PostDown directives that set routes and iptables rules for each Wireguard client individually. +- **Wireguard**: There is a one-to-one mapping between each Wireguard client and server instance listening on Mistborn. By design Wireguard clients cannot talk directly to each other but can use shared services and resources on Mistborn (e.g. Syncthing, Nextcloud, Jitisi, etc.) +- **Metrics**: In addition to the iptables INPUT policy set to DROP, an iptables chain exists that logs the packet meta data before dropping it. Mistborn redirects packets that will be dropped to this chain instead. A summary of the data about these dropped packets (unsolicited network traffic) can be found on the Metrics page. +- **Coppercloud**: Coppercloud works by populating ipsets with the ipset module in iptables to DROP (blacklist) or ACCEPT (whitelist) a given set of IP addresses. Upon system startup a celery task will compile the IP addresses, create the ipsets, and iptables rules. + +## Additonal Notes +- Interface names are not hardcoded anywhere in Mistborn. Two commands that are used in different circumstances to determine the default network interface and the interface that would route a public IP address are: `ip -o -4 route show to default` and `ip -o -4 route get 1.1.1.1`. +- The "Update" button will pull updated Docker images for mistborn, postgresql, redis, pihole, and dnscrypt. Those services will then be restarted. +- The generated TLS certificate has an RSA modulus of 4096 bits, is signed with SHA-256, and is good for 10 years. The nanny at Apple has decided to restrict the kinds of certificates iOS users may choose to manually trust and so you may have issues with TLS on an Apple device for now. + +# Roadmap +Many features and refinements are in the works at various stages including: + +- Option to upload metrics information to Cyber5K to refine each Mistborn instance's firewall +- Option to email default admin Wireguard config file +- Adding more extra services (e.g. Gitlab, Game Servers, etc.) +- Cyber5K marketplace to share Gateway access (to fixed IP addresses or domains, and for a fixed amount of time) +- Mistborn managing wireless interfaces for local access points (stripped down RaspAP) +- Optional periodic backup of local Mistborn config files and credentials to Cyber5K +- Internal network scan tool and feedback +- Anomaly detection in network traffic + # Contact Contact me at [steven@cyber5k.com](mailto:steven@cyber5k.com) From d523e87bb4296ae7a09de1ec936f63aef0f854c6 Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Sat, 25 Apr 2020 08:25:46 -0400 Subject: [PATCH 4/9] README: additional security notes --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 680d166..d80a835 100644 --- a/README.md +++ b/README.md @@ -273,7 +273,7 @@ These are some notes regarding the technical design and implementations of Mistb ## Attack Surface - **Wireguard**: Wireguard is the only way in to Mistborn. When new Wireguard profiles are generated they are attached to a random UDP port. Wireguard does not respond to unauthenticated traffic. External probes on the active Wireguard listening ports are not logged and do not appear on the Metrics page. -- **SSH**: If Mistborn is installed over SSH (most common) then an iptables rule is added allowing future SSH connections from the same source IP address. All other external SSH is blocked. Internal SSH (over the Wireguard tunnels) is allowed. +- **SSH**: If Mistborn is installed over SSH (most common) then an iptables rule is added allowing future SSH connections from the same source IP address. All other external SSH is blocked. Internal SSH (over the Wireguard tunnels) is allowed. Password authentication is disabled. The SSH key for the `mistborn` user is only accepted from internal source IP addresses. Fail2ban is also installed. - **Traefik**: Iptables closes web ports (TCP 80 and 443) from external access and additonally all web interfaces are behind the Traefik reverse-proxy. All web requests (e.g. home.mistborn) must be resolved by Mistborn DNS (Pihole/dnsmasq) and originate from a Wireguard tunnel. - **Docker**: When Docker exposes a port it creates a PREROUTING rule in the NAT table to catch eligible network requests. This means that even if your INPUT chain policy is DROP, your docker containers with exposed ports can receive and respond to traffic. Whenever Mistborn brings up a docker container with an exposed port it creates an iptables rule to block external traffic to that service. @@ -288,6 +288,7 @@ These are some notes regarding the technical design and implementations of Mistb - Interface names are not hardcoded anywhere in Mistborn. Two commands that are used in different circumstances to determine the default network interface and the interface that would route a public IP address are: `ip -o -4 route show to default` and `ip -o -4 route get 1.1.1.1`. - The "Update" button will pull updated Docker images for mistborn, postgresql, redis, pihole, and dnscrypt. Those services will then be restarted. - The generated TLS certificate has an RSA modulus of 4096 bits, is signed with SHA-256, and is good for 10 years. The nanny at Apple has decided to restrict the kinds of certificates iOS users may choose to manually trust and so you may have issues with TLS on an Apple device for now. +- Outbound UDP on port 53 is blocked. All DNS requests should be handled by the dnscrypt_proxy service and if any client, service, etc. tries to circumvent that it is blocked. # Roadmap Many features and refinements are in the works at various stages including: @@ -308,4 +309,5 @@ Contact me at [steven@cyber5k.com](mailto:steven@cyber5k.com) # Support Please consider supporting the project via: +- [Paypal.me](https://paypal.me/cyber5k) - [Patreon](https://www.patreon.com/cyber5k) From 4a1e577d65bc0bcddaadcdf6003d8765e3941a26 Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Sun, 26 Apr 2020 06:11:47 +0000 Subject: [PATCH 5/9] Resolve "DNS issue on Ubuntu 20.04" --- .gitignore | 1 + .gitlab-ci.yml | 1 + .gitmodules | 3 +++ README.md | 4 ++-- base.yml | 16 +++++++++------- modules/mistborn-cli | 1 + scripts/env/setup.sh | 21 +++++++++++++++++++++ scripts/install.sh | 11 ++++++++++- scripts/services/Mistborn-base.service | 1 + scripts/services/Mistborn-setup.service | 10 ++++++++++ scripts/subinstallers/docker.sh | 3 +++ scripts/subinstallers/docker_manual.sh | 3 --- scripts/update.sh | 18 ++++++++++++++++-- 13 files changed, 78 insertions(+), 15 deletions(-) create mode 100644 .gitmodules create mode 160000 modules/mistborn-cli create mode 100755 scripts/env/setup.sh create mode 100644 scripts/services/Mistborn-setup.service diff --git a/.gitignore b/.gitignore index ea8c8cf..0baf21e 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ compose/production/traefik/traefik.toml .envs/ +.env diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f56b619..03623c1 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -6,6 +6,7 @@ services: variables: GIT_SUBMODULE_STRATEGY: "recursive" MISTBORN_DEFAULT_PASSWORD: "default_password" + MISTBORN_DNS_BIND_IP: "10.2.3.1" stages: - test diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..067e6f8 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "modules/mistborn-cli"] + path = modules/mistborn-cli + url = https://gitlab.com/cyber5k/mistborn-cli.git diff --git a/README.md b/README.md index d80a835..913f02c 100644 --- a/README.md +++ b/README.md @@ -57,7 +57,7 @@ sudo bash ./mistborn/scripts/install.sh Get default admin Wireguard profile *wait 1 minute after "Mistborn Installed" message* ``` -sudo docker-compose -f /opt/mistborn/base.yml run --rm django python manage.py getconf admin default +sudo mistborn-cli getconf ``` Connect via Wireguard then visit `http://home.mistborn` @@ -130,7 +130,7 @@ sudo journalctl -xfu Mistborn-base The default Wireguard configuration file for `admin` may be obtained via: ``` -sudo docker-compose -f /opt/mistborn/base.yml run --rm django python manage.py getconf admin default +sudo mistborn-cli getconf ``` Please notice that the following lines are **NOT** part of the Wireguard config: ``` diff --git a/base.yml b/base.yml index 422c966..19fd8a6 100644 --- a/base.yml +++ b/base.yml @@ -7,7 +7,7 @@ volumes: services: django: - image: cyber5k/mistborn:latest + image: "cyber5k/mistborn:${MISTBORN_TAG}" container_name: mistborn_production_django depends_on: - postgres @@ -60,7 +60,7 @@ services: restart: unless-stopped celeryworker: - image: cyber5k/mistborn:latest + image: "cyber5k/mistborn:${MISTBORN_TAG}" container_name: mistborn_production_celeryworker volumes: - /home/mistborn/.ssh:/ssh @@ -83,7 +83,7 @@ services: restart: unless-stopped celeryworker-low-priority: - image: cyber5k/mistborn:latest + image: "cyber5k/mistborn:${MISTBORN_TAG}" container_name: mistborn_production_celeryworker_low_priority volumes: - /home/mistborn/.ssh:/ssh @@ -106,7 +106,7 @@ services: restart: unless-stopped celerybeat: - image: cyber5k/mistborn:latest + image: "cyber5k/mistborn:${MISTBORN_TAG}" container_name: mistborn_production_celerybeat env_file: - ./.envs/.production/.django @@ -116,7 +116,7 @@ services: flower: - image: cyber5k/mistborn:latest + image: "cyber5k/mistborn:${MISTBORN_TAG}" container_name: mistborn_production_flower env_file: - ./.envs/.production/.django @@ -129,9 +129,11 @@ services: pihole: container_name: mistborn_production_pihole image: pihole/pihole:latest + env_file: + - /opt/mistborn_volumes/base/base.txt ports: - - "53:53/tcp" - - "53:53/udp" + - "${MISTBORN_DNS_BIND_IP}:53:53/tcp" + - "${MISTBORN_DNS_BIND_IP}:53:53/udp" labels: - "traefik.enable=true" environment: diff --git a/modules/mistborn-cli b/modules/mistborn-cli new file mode 160000 index 0000000..00986bc --- /dev/null +++ b/modules/mistborn-cli @@ -0,0 +1 @@ +Subproject commit 00986bcb7f945c611d13099672b4fb1e2d0721f2 diff --git a/scripts/env/setup.sh b/scripts/env/setup.sh new file mode 100755 index 0000000..6b03c5a --- /dev/null +++ b/scripts/env/setup.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +VAR_FILE=/opt/mistborn/.env + +source /opt/mistborn/scripts/subinstallers/platform.sh + +MISTBORN_DNS_BIND_IP="0.0.0.0" +if [ "$DISTRO" == "ubuntu" ] && [ "$VERSION_ID" == "20.04" ]; then + MISTBORN_DNS_BIND_IP="10.2.3.1" +fi + +echo "MISTBORN_DNS_BIND_IP=${MISTBORN_DNS_BIND_IP}" | sudo tee ${VAR_FILE} +sudo chown mistborn:mistborn ${VAR_FILE} + +GIT_BRANCH=$(git -C /opt/mistborn symbolic-ref --short HEAD || echo "master") +MISTBORN_TAG="latest" +if [ "$GIT_BRANCH" != "master" ]; then + MISTBORN_TAG="test" +fi + +echo "MISTBORN_TAG=$MISTBORN_TAG" | sudo tee -a ${VAR_FILE} diff --git a/scripts/install.sh b/scripts/install.sh index 6857ea7..cb73dbf 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -148,6 +148,10 @@ then source ./scripts/subinstallers/cockpit.sh fi +# Mistborn-cli (pip3 installed by docker) +figlet "Mistborn: Installing mistborn-cli" +sudo pip3 install -e ./modules/mistborn-cli + # Mistborn # final setup vars iface=$(ip -o -4 route show to default | egrep -o 'dev [^ ]*' | awk 'NR==1{print $2}') @@ -165,6 +169,7 @@ fi sudo docker volume rm -f mistborn_production_postgres_data 2>/dev/null || true sudo docker volume rm -f mistborn_production_postgres_data_backups 2>/dev/null || true sudo docker volume rm -f mistborn_production_traefik 2>/dev/null || true +sudo docker volume prune -f 2>/dev/null || true # generate production .env file if [ ! -d ./.envs/.production ]; then @@ -207,6 +212,10 @@ source ./scripts/subinstallers/openssl.sh sudo rm -rf ../mistborn_volumes/base/tls sudo mv ./tls ../mistborn_volumes/base/ +# enable and run setup to generate .env +sudo systemctl enable Mistborn-setup.service +sudo systemctl start Mistborn-setup.service + # Download docker images while DNS is operable sudo docker-compose -f base.yml pull || true sudo docker-compose -f base.yml build @@ -240,4 +249,4 @@ popd figlet "Mistborn Installed" echo "Watch Mistborn start: sudo journalctl -xfu Mistborn-base" -echo "Retrieve Wireguard default config for admin: sudo docker-compose -f /opt/mistborn/base.yml run --rm django python manage.py getconf admin default" +echo "Retrieve Wireguard default config for admin: sudo mistborn-cli getconf" diff --git a/scripts/services/Mistborn-base.service b/scripts/services/Mistborn-base.service index b1c52b5..a810c4a 100644 --- a/scripts/services/Mistborn-base.service +++ b/scripts/services/Mistborn-base.service @@ -9,6 +9,7 @@ User=root Group=docker PermissionsStartOnly=true # Shutdown container (if running) when unit is stopped +EnvironmentFile=/opt/mistborn/.env ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/base.yml down ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/base.yml build diff --git a/scripts/services/Mistborn-setup.service b/scripts/services/Mistborn-setup.service new file mode 100644 index 0000000..9b83c71 --- /dev/null +++ b/scripts/services/Mistborn-setup.service @@ -0,0 +1,10 @@ +[Unit] +Description=Mistborn Environment Setup +Before=Mistborn-base.service + +[Service] +Type=oneshot +ExecStart=/opt/mistborn/scripts/env/setup.sh + +[Install] +WantedBy=multi-user.target diff --git a/scripts/subinstallers/docker.sh b/scripts/subinstallers/docker.sh index 1542207..b21d452 100755 --- a/scripts/subinstallers/docker.sh +++ b/scripts/subinstallers/docker.sh @@ -1,5 +1,8 @@ #!/bin/bash +# Docker +figlet "Mistborn: Installing Docker" + sudo apt update sudo apt install -y python python3-pip python3-setuptools libffi-dev python3-dev libssl-dev diff --git a/scripts/subinstallers/docker_manual.sh b/scripts/subinstallers/docker_manual.sh index 176aba9..9692109 100755 --- a/scripts/subinstallers/docker_manual.sh +++ b/scripts/subinstallers/docker_manual.sh @@ -1,8 +1,5 @@ #!/bin/bash -# Docker -figlet "Mistborn: Installing Docker" - # dependencies echo "Installing Docker dependencies" sudo apt-get install -y \ diff --git a/scripts/update.sh b/scripts/update.sh index 8f3d639..049fbec 100755 --- a/scripts/update.sh +++ b/scripts/update.sh @@ -2,7 +2,21 @@ set -e -sudo docker-compose -f /opt/mistborn/base.yml pull -sudo docker-compose -f /opt/mistborn/base.yml build +sudo git -C /opt/mistborn pull +sudo git -C /opt/mistborn submodule update --init --recursive + +# ensure mistborn-cli is installed +sudo pip3 install -e /opt/mistborn/modules/mistborn-cli + +iface=$(ip -o -4 route show to default | egrep -o 'dev [^ ]*' | awk 'NR==1{print $2}') +sudo cp /opt/mistborn/scripts/services/Mistborn* /etc/systemd/system/ +sudo find /etc/systemd/system/ -type f -name 'Mistborn*' | xargs sudo sed -i "s/User=root/User=mistborn/" +sudo find /etc/systemd/system/ -type f -name 'Mistborn*' | xargs sudo sed -i "s/DIFACE/$iface/" + +sudo systemctl daemon-reload +sudo systemctl enable Mistborn-setup.service +sudo systemctl restart Mistborn-setup.service + +sudo mistborn-cli pullbuild sudo systemctl restart Mistborn-base From 66e5d1879d9c90e4059d122a7ec0c2f1d3928821 Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Mon, 27 Apr 2020 02:53:54 +0000 Subject: [PATCH 6/9] Resolve "Raspbian" --- README.md | 2 +- {dev => scripts/env}/wg_clean.sh | 0 scripts/install.sh | 32 +++++++++++++++++++------------- 3 files changed, 20 insertions(+), 14 deletions(-) rename {dev => scripts/env}/wg_clean.sh (100%) diff --git a/README.md b/README.md index 913f02c..8347006 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # Mistborn -A platform for easily managing your cloud server and Wireguard access +A platform for easily standing up and managing your own cloud services, with firewall, ad-blocking, and Wireguard access # Table of Contents [[_TOC_]] diff --git a/dev/wg_clean.sh b/scripts/env/wg_clean.sh similarity index 100% rename from dev/wg_clean.sh rename to scripts/env/wg_clean.sh diff --git a/scripts/install.sh b/scripts/install.sh index cb73dbf..7da2197 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -120,6 +120,7 @@ sudo sed -i 's/#PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/ss sudo sed -i 's/PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config sudo sed -i 's/#PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config sudo sed -i 's/PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config +sudo systemctl enable ssh sudo systemctl restart ssh # Additional tools fail2ban @@ -137,6 +138,8 @@ source ./scripts/subinstallers/wireguard.sh # Docker source ./scripts/subinstallers/docker.sh +sudo systemctl enable docker +sudo systemctl start docker # Unattended upgrades sudo apt-get install -y unattended-upgrades @@ -160,21 +163,11 @@ figlet "Mistborn default NIC: $iface" #IPV4_PUBLIC=$(ip -o -4 route show default | egrep -o 'dev [^ ]*' | awk '{print $2}' | xargs ip -4 addr show | grep 'inet ' | awk '{print $2}' | grep -o "^[0-9.]*" | tr -cd '\11\12\15\40-\176' | head -1) # tail -1 to get last IPV4_PUBLIC="10.2.3.1" -# clean -if [ -f "/etc/systemd/system/Mistborn-base.service" ]; then - sudo systemctl stop Mistborn*.service 2>/dev/null || true - sudo systemctl disable Mistborn*.service 2>/dev/null || true -fi - -sudo docker volume rm -f mistborn_production_postgres_data 2>/dev/null || true -sudo docker volume rm -f mistborn_production_postgres_data_backups 2>/dev/null || true -sudo docker volume rm -f mistborn_production_traefik 2>/dev/null || true -sudo docker volume prune -f 2>/dev/null || true # generate production .env file -if [ ! -d ./.envs/.production ]; then - ./scripts/subinstallers/gen_prod_env.sh "$MISTBORN_DEFAULT_PASSWORD" -fi +#if [ ! -d ./.envs/.production ]; then +./scripts/subinstallers/gen_prod_env.sh "$MISTBORN_DEFAULT_PASSWORD" +#fi # unattended upgrades sudo cp ./scripts/conf/20auto-upgrades /etc/apt/apt.conf.d/ @@ -242,6 +235,19 @@ echo "backup up original volumes folder" sudo mkdir -p ../mistborn_backup sudo tar -czf ../mistborn_backup/mistborn_volumes_backup.tar.gz ../mistborn_volumes 1>/dev/null 2>&1 +# clean docker +echo "cleaning old docker volumes" +sudo systemctl stop Mistborn-base || true +sudo docker-compose -f /opt/mistborn/base.yml kill +sudo docker volume rm -f mistborn_production_postgres_data 2>/dev/null || true +sudo docker volume rm -f mistborn_production_postgres_data_backups 2>/dev/null || true +sudo docker volume rm -f mistborn_production_traefik 2>/dev/null || true +sudo docker volume prune -f 2>/dev/null || true + +# clean Wireguard +echo "cleaning old wireguard services" +sudo ./scripts/env/wg_clean.sh + # start base service sudo systemctl enable Mistborn-base.service sudo systemctl start Mistborn-base.service From 38bc45300b560330af1aa579bd67d6ba90dd572a Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Tue, 28 Apr 2020 02:34:15 +0000 Subject: [PATCH 7/9] Resolve "Docs for Ubuntu Upgrade" --- README.md | 63 ++++++++++++++++++++++++++++++++++++++++------ scripts/install.sh | 5 ++-- 2 files changed, 58 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 8347006..a47371b 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # Mistborn -A platform for easily standing up and managing your own cloud services, with firewall, ad-blocking, and Wireguard access +A secure platform for easily standing up and managing your own cloud services: including firewall, ad-blocking, and Wireguard VPN access # Table of Contents [[_TOC_]] @@ -7,12 +7,21 @@ A platform for easily standing up and managing your own cloud services, with fir # What is Mistborn The term [Mistborn](http://www.brandonsanderson.com/the-mistborn-saga-the-original-trilogy) is inspired by a type of powerful Allomancer in Brandon Sanderson's Cosmere. -Mistborn started as a passion project for a husband and father protecting his family. Certain family members insisted on connecting their devices to free WiFi networks. We needed a way to secure all family devices with a solid VPN (Wireguard). Once we had that we wanted to control DNS to block ads to all devices and block malicious and pornographic websites across all family devices. Then we wanted chat, file-sharing, and webchat services that we could use for ourselves without entrusting our data to some big tech company. And then... home automation. I know I'll be adding services as I go so I made that easy to do. +Mistborn started as a passion project for a husband and father protecting his family. Certain family members insisted on connecting their devices to free public WiFi networks. We needed a way to secure all family devices with a solid VPN (Wireguard). Once we had that we wanted to control DNS to block ads to all devices and block malicious websites across all family devices. Then we wanted chat, file-sharing, and webchat services that we could use for ourselves without entrusting our data to some big tech company. And then... home automation. I know I'll be adding more services so I made that easy to do. -Mistborn depends on these open source technologies: +Ideal for teams who: +- hate internet ads +- need to be protected from malicious internet domains +- need to collaborate securely +- want to retain sole ownership of their data +- want to easily grant and revoke access to people and devices via an easy web interface +- want secure internet access wherever they are +- want to limit or stop data-collecting services + +Mistborn depends on these core open source technologies: - [Docker](https://www.docker.com/why-docker): containerization - [Wireguard](https://www.wireguard.com): secure VPN access -- [SSH](https://www.openssh.com): secure password-less remote management +- [SSH](https://www.openssh.com): secure remote management These tools are not vital to Mistborn itself but are integrated to enhance security, ease, and features: - [iptables](https://www.netfilter.org): The powerful Linux netfilter firewall tool @@ -80,13 +89,13 @@ Pihole provides a way to block outgoing DNS requests for given lists of blocked This example shows Coppercloud blocking a list of Microsoft IP addresses on a network with Windows 10 clients. # Gateways -I was getting frustrated at being forced to choose between being connected to my VPN and using streaming services that I have paid for. +We were getting frustrated at being forced to choose between being connected to our VPN and using streaming services that we have paid for. ![Netflix blocked](https://gitlab.com/cyber5k/public/-/raw/master/graphics/netflix_blocked.png) *Netflix blocking my connections that it sees coming from a DigitalOcean droplet* -In Mistborn, Gateways are upstream from the VPN server so connections to third-party services (e.g. Netflix, Hulu, etc.) will appear to be coming from the public IP address of the Gateway. I setup a Gateway at home, then all VPN profiles created with this Gateway will apear to be coming from my house and are not blocked. No port-forwarding required (assuming Mistborn is publicly accessible). +In Mistborn, Gateways are upstream from the VPN server so connections to third-party services (e.g. Netflix, Hulu, etc.) will appear to be coming from the public IP address of the Gateway. I setup a Gateway at home (Mistborn on DigitalOcean) then all Wireguard profiles created with this Gateway will appear to be coming from my house and are not blocked. No port-forwarding required (assuming Mistborn is publicly accessible). ![Mistborn Gateway Diagram](https://gitlab.com/cyber5k/public/-/raw/master/graphics/gateway_network.png) @@ -227,6 +236,35 @@ On Gateway: - Run `sudo systemctl start wg-quick@gateway` - Run `sudo systemctl enable wg-quick@gateway` +# FAQ +Frequently Asked Questions + +## Where is My Data? + +The Docker services mount volumes located in: +``` +/opt/mistborn_volumes +``` + +The core Mistborn services have volumes mounted in `/opt/mistborn_volumes/base`. These should not be modified. The extra services' volumes are mounted in: +``` +/opt/mistborn_volumes/extra +``` + +Your data from Nextcloud, Syncthing, Bitwarden, etc. will be located there. + +## How do I SSH into Mistborn? +If Mistborn is installed via SSH then an iptables rule is added allowing external SSH connections from the same source IP address only. If Mistborn was installed locally then no external SSH is permitted. + +SSH is permitted from any device connected to Mistborn by Wireguard. + +Password authentication in enabled. Mistborn disables password authentication for root. Fail2ban blocks IPs with excessive failed login attempts. + +You can SSH using the Mistborn domain when connected by Wireguard: +``` +ssh user@home.mistborn +``` + # Troubleshooting Once you're connected to Wireguard you should see .mistborn domains and the internet should work as expected. Be sure to use http (http://home.mistborn). Wireguard is the encrypted channel so there's usually no need to bother with TLS certs (WebRTC functionality and some mobile apps require TLS so it is available). Here are some things to check if you have issues: @@ -268,12 +306,23 @@ Be sure to restart Docker afterward: sudo systemctl restart docker ``` +## Troubleshooting Upgrading from Ubuntu 18.04 to 20.04 +New installations of 18.04 and 20.04 after 25 April 2020 don't seem to be having issues. If you installed Mistborn on Ubuntu 18.04 prior to 25 April 2020 and then upgrade to 20.04 you may have one minor issue described below. + +Owing to changes in docker NAT rules and container DNS resolution, some Wireguard client configurations generated with Mistborn before 25 April 2020 (be sure to update Mistborn) may experience issues after upgrading to Ubuntu 20.04 LTS. Symptoms: can ping but can't resolve DNS. + +Solution: Edit the Wireguard client config and set the DNS directive as follows: +``` +DNS = 10.2.3.1 +``` +Close the config and restart the client Wireguard process. + # Technical and Security Insights These are some notes regarding the technical design and implementations of Mistborn. Feel free to contact me for additional details. ## Attack Surface - **Wireguard**: Wireguard is the only way in to Mistborn. When new Wireguard profiles are generated they are attached to a random UDP port. Wireguard does not respond to unauthenticated traffic. External probes on the active Wireguard listening ports are not logged and do not appear on the Metrics page. -- **SSH**: If Mistborn is installed over SSH (most common) then an iptables rule is added allowing future SSH connections from the same source IP address. All other external SSH is blocked. Internal SSH (over the Wireguard tunnels) is allowed. Password authentication is disabled. The SSH key for the `mistborn` user is only accepted from internal source IP addresses. Fail2ban is also installed. +- **SSH**: If Mistborn is installed over SSH (most common) then an iptables rule is added allowing future SSH connections from the same source IP address. All other external SSH is blocked. Internal SSH (over the Wireguard tunnels) is allowed. Password authentication is allowed. The SSH key for the `mistborn` user is only accepted from internal source IP addresses. Fail2ban is also installed. - **Traefik**: Iptables closes web ports (TCP 80 and 443) from external access and additonally all web interfaces are behind the Traefik reverse-proxy. All web requests (e.g. home.mistborn) must be resolved by Mistborn DNS (Pihole/dnsmasq) and originate from a Wireguard tunnel. - **Docker**: When Docker exposes a port it creates a PREROUTING rule in the NAT table to catch eligible network requests. This means that even if your INPUT chain policy is DROP, your docker containers with exposed ports can receive and respond to traffic. Whenever Mistborn brings up a docker container with an exposed port it creates an iptables rule to block external traffic to that service. diff --git a/scripts/install.sh b/scripts/install.sh index 7da2197..87f4e2f 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -72,7 +72,6 @@ if [ -z "${MISTBORN_INSTALL_COCKPIT}" ]; then MISTBORN_INSTALL_COCKPIT=${MISTBORN_INSTALL_COCKPIT:-Y} fi - # SSH keys if [ ! -f ~/.ssh/id_rsa ]; then echo "Generating SSH keypair for $USER" @@ -116,8 +115,8 @@ fi # SSH Server sudo apt-get install -y openssh-server -sudo sed -i 's/#PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config -sudo sed -i 's/PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config +sudo sed -i 's/#PasswordAuthentication.*/PasswordAuthentication yes/' /etc/ssh/sshd_config +sudo sed -i 's/PasswordAuthentication.*/PasswordAuthentication yes/' /etc/ssh/sshd_config sudo sed -i 's/#PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config sudo sed -i 's/PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config sudo systemctl enable ssh From 7bf28772bcb2c2c9fba92e84ee326b0f53c7d07f Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Mon, 27 Apr 2020 22:45:02 -0400 Subject: [PATCH 8/9] README: edits --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index a47371b..d0aeb0e 100644 --- a/README.md +++ b/README.md @@ -14,9 +14,9 @@ Ideal for teams who: - need to be protected from malicious internet domains - need to collaborate securely - want to retain sole ownership of their data -- want to easily grant and revoke access to people and devices via an easy web interface +- want to easily grant and revoke access to people and devices via a simple web interface - want secure internet access wherever they are -- want to limit or stop data-collecting services +- want to limit or stop data collecting services Mistborn depends on these core open source technologies: - [Docker](https://www.docker.com/why-docker): containerization @@ -306,7 +306,7 @@ Be sure to restart Docker afterward: sudo systemctl restart docker ``` -## Troubleshooting Upgrading from Ubuntu 18.04 to 20.04 +## Troubleshooting Upgrade from Ubuntu 18.04 to 20.04 New installations of 18.04 and 20.04 after 25 April 2020 don't seem to be having issues. If you installed Mistborn on Ubuntu 18.04 prior to 25 April 2020 and then upgrade to 20.04 you may have one minor issue described below. Owing to changes in docker NAT rules and container DNS resolution, some Wireguard client configurations generated with Mistborn before 25 April 2020 (be sure to update Mistborn) may experience issues after upgrading to Ubuntu 20.04 LTS. Symptoms: can ping but can't resolve DNS. From d543e7437bb2d1c534d20e335a7ae943a654394d Mon Sep 17 00:00:00 2001 From: Steven Foerster Date: Wed, 29 Apr 2020 12:23:10 +0000 Subject: [PATCH 9/9] Resolve "README: troubleshooting extra services" --- .gitignore | 1 + README.md | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 63 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index 0baf21e..86e524d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ compose/production/traefik/traefik.toml .envs/ .env +*.swp diff --git a/README.md b/README.md index d0aeb0e..38db946 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,7 @@ Ideal for teams who: - want to easily grant and revoke access to people and devices via a simple web interface - want secure internet access wherever they are - want to limit or stop data collecting services +- want to prevent being detected/blocked for using a proxy or VPN service Mistborn depends on these core open source technologies: - [Docker](https://www.docker.com/why-docker): containerization @@ -48,13 +49,22 @@ Tested Operating Systems (in order of thoroughness): - Debian 10 (Buster) - Raspbian Buster +The default tests are run on DigitalOcean Droplets: 2GB RAM, 1 CPU, 50GB hard disk. + +The Mistborn docker images exist for these architectures: + +| Mistborn Docker Images (hub.docker.com) | Architectures | +|------------------------------------------------|---------------------| +| mistborn (django, celery{worker,beat}, flower) | amd64, arm64, armv7 | +| dnscrypt-proxy | amd64, arm64, armv7 | + Recommended System Specifications: | Use Case | Description | RAM | Hard Disk | |------------------------|-------------------------------------------------------------------------------|-------|-----------| -| Bare bones | Wireguard, Pihole (no Cockpit, no extra services) | 1 GB | 10 GB | -| Default | Bare bones + Cockpit | 2 GB | 10 GB | -| Low-resource services | Default + Bitwarden, Tor, Syncthing | 3 GB | 15 GB | +| Bare bones | Wireguard, Pihole (no Cockpit, no extra services) | 1 GB | 15 GB | +| Default | Bare bones + Cockpit | 2 GB | 15 GB | +| Low-resource services | Default + Bitwarden, Tor, Syncthing | 3 GB | 20 GB | | High-resource services | Default + Jitsi, Nextcloud, Jellyfin, Rocket.Chat, Home Assistant, OnlyOffice | 4 GB+ | 25 GB+ | Starting from base installation @@ -236,6 +246,41 @@ On Gateway: - Run `sudo systemctl start wg-quick@gateway` - Run `sudo systemctl enable wg-quick@gateway` +# Phones and Mobile Devices +All your devices can be connected to Mistborn as Wireguard clients. + +First steps: +1. Device: Download the Wireguard app on your device. Links: [Android](https://play.google.com/store/apps/details?id=com.wireguard.android) [Apple](https://apps.apple.com/us/app/wireguard/id1441195209) +1. Mistborn: Create a Wireguard profile for the device. +1. Device: Scan Wireguard client QR code in Wireguard app. +1. Device: Enable Wireguard connection. + +All of you device network traffic is now being routed through Wireguard. Ads and malicious sites are blocked by pihole. DNS queries are verified via DNScrypt. + +But wait, there's more! You can: +- visit the [Mistborn web interface](http://home.mistborn) through your phone's browser. +- download the apps for any extra services you have running and connect them to your Mistborn using the Mistborn domains. + +## App Links + +| | Android | Apple | +|----------------|----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------| +| Nextcloud | [Nextcloud](https://play.google.com/store/apps/details?id=com.nextcloud.client) | [Nextcloud](https://apps.apple.com/us/app/nextcloud/id1125420102) | +| Syncthing | [Syncthing](https://play.google.com/store/apps/details?id=com.nutomic.syncthingandroid) | | +| Jitsi Meet | [Jitsi Meet](https://play.google.com/store/apps/details?id=org.jitsi.meet) | [Jitsi Meet](https://apps.apple.com/us/app/jitsi-meet/id1165103905) | +| Bitwarden | [Bitwarden](https://play.google.com/store/apps/details?id=com.x8bit.bitwarden) | [Bitwarden](https://apps.apple.com/us/app/bitwarden-password-manager/id1137397744) | +| Jellyfin | [Jellyfin](https://play.google.com/store/apps/details?id=org.jellyfin.mobile) | [Jellyfin](https://apps.apple.com/us/app/jellyfin-mobile/id1480192618) | +| Home Assistant | [Home Assistant](https://play.google.com/store/apps/details?id=io.homeassistant.companion.android) | | +| Rocket.Chat | [Rocket.Chat](https://play.google.com/store/apps/details?id=chat.rocket.android) | [Rocket.Chat](https://apps.apple.com/us/app/rocket-chat/id1148741252) | + +## TLS Certificate +Some apps require TLS (HTTPS). All traffic to Mistborn domains already occurs over Wireguard but to keep apps running, a TLS certificate exists for Mistborn and can be imported into your device's trusted credentials in the security settings. + +The TLS certificate can be found here: +``` +/opt/mistborn_volumes/base/tls/cert.crt +``` + # FAQ Frequently Asked Questions @@ -295,6 +340,20 @@ The `dev/` folder contains a script for completing a hard reset: destroying and sudo ./dev/rebuild.sh ``` +## Troubleshooting Extra Services +Each extra service has its own systemd process which can be monitored: +``` +sudo journalctl -xfu Mistborn-homeassistant +sudo journalctl -xfu Mistborn-bitwarden +sudo journalctl -xfu Mistborn-syncthing +sudo journalctl -xfu Mistborn-jellyfin +sudo journalctl -xfu Mistborn-nextcloud +sudo journalctl -xfu Mistborn-jitsi +sudo journalctl -xfu Mistborn-rocketchat +sudo journalctl -xfu Mistborn-onlyoffice +sudo journalctl -xfu Mistborn-tor +``` + ## Troubleshooting Docker Instead of defaulting to a system DNS server, Docker will try to use a public DNS server (e.g. 8.8.8.8). If you're having issues pulling or building Docker containers with "failure to connect" errors, this is the likely problem. You can manually set the DNS server Docker should use with the `DOCKER_OPTS` field in `/etc/default/docker`. Example: ```