PUID and GUID

syncing with docker image: public IP selection

Closes #82

See merge request cyber5k/mistborn!43

README.md

@@ -42,6 +42,8 @@ Within Mistborn is a panel to enable and manage these free extra services (off b
 - [Jellyfin](https://jellyfin.org): The Free Media Software System.
 - [Tor](https://www.torproject.org): The Onion Router. One tool in the arsenal of online security and privacy.
 - [Jitsi](https://jitsi.org): Multi-platform open-source video conferencing
+- [Mattermost](https://mattermost.com/): A flexible, open source messaging platform that enables secure team collaboration
+
 
 # Quickstart
 Tested Operating Systems (in order of thoroughness):
@@ -68,10 +70,10 @@ Recommended System Specifications:
 
 | Use Case               | Description                                                                   | RAM   | Hard Disk |
 |------------------------|-------------------------------------------------------------------------------|-------|-----------|
-| Bare bones             | Wireguard, Pihole (no Cockpit, no extra services)                             | 1 GB  | 15 GB     |
+| Bare bones             | Wireguard, Pihole (no Cockpit, no extra services)                             | 2 GB  | 15 GB     |
-| Default                | Bare bones + Cockpit                                                          | 2 GB  | 15 GB     |
+| Default                | Bare bones + Cockpit                                                          | 2 GB+ | 15 GB     |
-| Low-resource services  | Default + Bitwarden, Tor, Syncthing                                           | 3 GB  | 20 GB     |
+| Low-resource services  | Default + Bitwarden, Tor, Syncthing                                           | 4 GB  | 20 GB     |
-| High-resource services | Default + Jitsi, Nextcloud, Jellyfin, Rocket.Chat, Home Assistant, OnlyOffice | 4 GB+ | 25 GB+    |
+| High-resource services | Default + Jitsi, Nextcloud, Jellyfin, Rocket.Chat, Home Assistant, OnlyOffice | 6 GB+ | 25 GB+    |
 
 Starting from base installation
 ```
@@ -111,7 +113,7 @@ We were getting frustrated at being forced to choose between being connected to
 
 *Netflix blocking my connections that it sees coming from a DigitalOcean droplet*
 
-In Mistborn, Gateways are upstream from the VPN server so connections to third-party services (e.g. Netflix, Hulu, etc.) will appear to be coming from the public IP address of the Gateway. I setup a Gateway at home (Mistborn on DigitalOcean) then all Wireguard profiles created with this Gateway will appear to be coming from my house and are not blocked. No port-forwarding required (assuming Mistborn is publicly accessible).
+In Mistborn, Gateways are upstream from the VPN server so connections to third-party services (e.g. Netflix, Hulu, etc.) will appear to be coming from the public IP address of the Gateway. I setup a Gateway at home (Raspberry Pi with `wireguard` and `openresolv` installed) and with our Mistborn on DigitalOcean, all Wireguard profiles created with this Gateway will appear to be coming from my house and are not blocked. No port-forwarding required (assuming Mistborn is publicly accessible).
 
 ![Mistborn Gateway Diagram](https://gitlab.com/cyber5k/public/-/raw/master/graphics/gateway_network.png)
 
@@ -260,6 +262,7 @@ Mistborn uses the following domains (that can be reached by all Wireguard client
 | Syncthing | syncthing.mistborn | Off |
 | OnlyOffice | onlyoffice.mistborn | Off |
 | Jitsi | jitsi.mistborn | Off |
+| Mattermost | mattermost.mistborn | Off |
 
 # Default Credentials
 These are the default credentials to use in the services you choose to use:
@@ -317,6 +320,7 @@ But wait, there's more! You can:
 | Jellyfin       | [Jellyfin](https://play.google.com/store/apps/details?id=org.jellyfin.mobile)                      | [Jellyfin](https://apps.apple.com/us/app/jellyfin-mobile/id1480192618)             |
 | Home Assistant | [Home Assistant](https://play.google.com/store/apps/details?id=io.homeassistant.companion.android) |                                                                                    |
 | Rocket.Chat    | [Rocket.Chat](https://play.google.com/store/apps/details?id=chat.rocket.android)                   | [Rocket.Chat](https://apps.apple.com/us/app/rocket-chat/id1148741252)              |
+| Mattermost     | [Mattermost](https://play.google.com/store/apps/details?id=com.mattermost.rn)                      | [Mattermost](https://apps.apple.com/us/app/mattermost/id1257222717)                |
 
 ## TLS Certificate
 Some apps require TLS (HTTPS). All traffic to Mistborn domains already occurs over Wireguard but to keep apps running, a TLS certificate exists for Mistborn and can be imported into your device's trusted credentials in the security settings. This certificate is checked every day and will be re-generated when expiration is less than 30 days away.
@@ -355,10 +359,41 @@ You can SSH using the Mistborn domain when connected by Wireguard:
 ssh user@home.mistborn
 ```
 
+## How do I change the upstream DNSCrypt servers?
+The upstream servers used by dnscrypt-proxy are set in:  
+
+`base.yml`:
+```
+services:
+...
+  dnscrypt-proxy:
+  ...
+    environment:
+    ...
+      - DNSCRYPT_SERVER_NAMES=[...]
+```
+
+The available options are here: https://download.dnscrypt.info/dnscrypt-resolvers/v2/public-resolvers.md
+
 # Troubleshooting
 
 Once you're connected to Wireguard you should see .mistborn domains and the internet should work as expected. Be sure to use http (http://home.mistborn). Wireguard is the encrypted channel so there's usually no need to bother with TLS certs (WebRTC functionality and some mobile apps require TLS so it is available). Here are some things to check if you have issues:
 
+Check if you can ping an external IP address:
+```
+ping 1.1.1.1
+```
+
+Check if you can resolve local DNS queries:
+```
+dig home.mistborn
+```
+
+Check if you can resolve external DNS queries:
+```
+dig cyber5k.com
+```
+
 See if any docker containers are stopped:
 ```
 sudo docker container ls -a
@@ -385,6 +420,9 @@ The `dev/` folder contains a script for completing a hard reset: destroying and
 sudo ./dev/rebuild.sh
 ```
 
+## Troubleshooting Wireguard
+Ensure that your public IP address in your client profile (e.g. `Endpoint = <Mistborn public IP address>:<random port>`) is actually publicly available (not in 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16) if you are attempting to access Mistborn across the internet.
+
 ## Troubleshooting Extra Services
 Each extra service has its own systemd process which can be monitored:
 ```
@@ -475,4 +513,5 @@ Contact me at [steven@cyber5k.com](mailto:steven@cyber5k.com)
 
 Please consider supporting the project via:
 - [Paypal.me](https://paypal.me/cyber5k)
+- [Buy me a drink](https://www.buymeacoffee.com/cyber5k)
 - [Patreon](https://www.patreon.com/cyber5k)

extra/mattermost.yml

@@ -0,0 +1,88 @@
+version: "3"
+
+services:
+
+  db:
+    #build:
+    #  context: ../compose/production/mattermost/db
+    image: mattermost/mattermost-prod-db
+    read_only: true
+    restart: unless-stopped
+    volumes:
+      - ../../mistborn_volumes/extra/mattermost/db/var/lib/postgresql/data:/var/lib/postgresql/data
+      - /etc/localtime:/etc/localtime:ro
+    #environment:
+    #  - POSTGRES_USER=mmuser
+    #  - POSTGRES_PASSWORD=mmuser_password
+    #  - POSTGRES_DB=mattermost
+    # uncomment the following to enable backup
+    #  - AWS_ACCESS_KEY_ID=XXXX
+    #  - AWS_SECRET_ACCESS_KEY=XXXX
+    #  - WALE_S3_PREFIX=s3://BUCKET_NAME/PATH
+    #  - AWS_REGION=us-east-1
+
+  app:
+    #build:
+    #  context: ../compose/production/mattermost/app
+    #  # uncomment following lines for team edition or change UID/GID
+    #  args:
+    #    - edition=team
+    #    - PUID=1000
+    #    - PGID=1000
+    image: mattermost/mattermost-team-edition
+    restart: unless-stopped
+    volumes:
+      - ../../mistborn_volumes/extra/mattermost/app/mattermost/config:/mattermost/config:rw
+      - ../../mistborn_volumes/extra/mattermost/app/mattermost/data:/mattermost/data:rw
+      - ../../mistborn_volumes/extra/mattermost/app/mattermost/logs:/mattermost/logs:rw
+      - ../../mistborn_volumes/extra/mattermost/app/mattermost/plugins:/mattermost/plugins:rw
+      - ../../mistborn_volumes/extra/mattermost/app/mattermost/client-plugins:/mattermost/client/plugins:rw
+      - /etc/localtime:/etc/localtime:ro
+      # When you want to use SSO with GitLab, you have to add the cert pki chain of GitLab inside Alpine
+      # to avoid Token request failed: certificate signed by unknown authority (link: https://github.com/mattermost/mattermost-server/issues/13059)
+      # - <path_to_your_gitlab_pki>/pki_chain.pem:/etc/ssl/certs/pki_chain.pem:ro
+    environment:
+    #  # set same as db credentials and dbname
+    #  #- MM_USERNAME=mmuser
+    #  #- MM_PASSWORD=mmuser_password
+    #  #- MM_DBNAME=mattermost
+
+    #  # use the credentials you've set above, in the format:
+    #  # MM_SQLSETTINGS_DATASOURCE=postgres://${MM_USERNAME}:${MM_PASSWORD}@db:5432/${MM_DBNAME}?sslmode=disable&connect_timeout=10
+    #  #- MM_SQLSETTINGS_DATASOURCE=postgres://mmuser:mmuser_password@db:5432/mattermost?sslmode=disable&connect_timeout=10
+      - MM_SQLSETTINGS_DATASOURCE=postgres://${MM_USERNAME}:${MM_PASSWORD}@db:5432/${MM_DBNAME}?sslmode=disable&connect_timeout=10
+      - PUID=2000
+      - GUID=2000
+
+    #  # in case your config is not in default location
+    #  #- MM_CONFIG=/mattermost/config/config.json
+
+  web:
+    #build:
+    #  context: ../compose/production/mattermost/web
+    image: mattermost/mattermost-prod-web
+    #ports:
+    #  - "80:80"
+    #  - "443:443"
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.mattermost-http.rule=Host(`mattermost.mistborn`)"
+      - "traefik.http.routers.mattermost-http.entrypoints=web"
+      - "traefik.http.routers.mattermost-http.middlewares=mistborn_auth@file"
+      - "traefik.http.routers.mattermost-https.rule=Host(`mattermost.mistborn`)"
+      - "traefik.http.routers.mattermost-https.entrypoints=websecure"
+      - "traefik.http.routers.mattermost-https.middlewares=mistborn_auth@file"
+      - "traefik.http.routers.mattermost-https.tls.certresolver=basic"
+      - "traefik.http.services.mattermost-service.loadbalancer.server.port=80"
+    read_only: true
+    restart: unless-stopped
+    volumes:
+      # This directory must have cert files if you want to enable SSL
+      #- ./volumes/web/cert:/cert:ro
+      - /etc/localtime:/etc/localtime:ro
+
+networks:
+  default:
+    external:
+      name: mistborn_default
+  

extra/rocketchat.yml

@@ -35,8 +35,8 @@ services:
     container_name: mistborn_production_rocketchat_mongo
     restart: unless-stopped
     volumes:
-     - ../volumes/extra/rocketchat/data/db:/data/db
+     - ../../mistborn_volumes/extra/rocketchat/data/db:/data/db
-     - ../volumes/extra/rocketchat/data/dump:/dump
+     - ../../mistborn_volumes/extra/rocketchat/data/dump:/dump
     command: mongod --smallfiles --oplogSize 128 --replSet rs0 --storageEngine=mmapv1
 
   # this container's job is just run the command to initialize the replica set.
@@ -61,7 +61,7 @@ services:
     depends_on:
       - rocketchat
     volumes:
-      - ../volumes/extra/rocketchat/hubot/scripts:/home/hubot/scripts
+      - ../../mistborn_volumes/extra/rocketchat/hubot/scripts:/home/hubot/scripts
   # this is used to expose the hubot port for notifications on the host on port 3001, e.g. for hubot-jenkins-notifier
     ports:
       - 3001:8080/tcp

extra/wifi.yml

@@ -1,23 +0,0 @@
-version: '3'
-
-services:
-  wifi:
-    #user: root
-    image: "cyber5k/wifi:${MISTBORN_TAG}"
-    container_name: mistborn_production_wifi
-    #labels:
-    #  - "traefik.enable=true"
-    #  - "traefik.port=80"
-    #env_file:
-    #  - ../.envs/.production/.pihole
-    #command: /start
-    volumes:
-      - ../../mistborn_volumes/extra/wifi/hostapd.conf:/etc/hostapd/hostapd.conf
-      - ../../mistborn_volumes/extra/wifi/dnsmasq.conf:/etc/dnsmasq.conf
-    #  - ../../mistborn_volumes/extra/wifi/wificfg.json:/cfg/wificfg.json
-    network_mode: bridge #host
-    cap_add:
-      - NET_ADMIN
-      - NET_RAW
-    #privileged: true
-

scripts/conf/wificfg.json

@@ -1,18 +0,0 @@
-{
-    "wifi_iface": "wlan0",
-    "dnsmasq_cfg": {
-	"address": "/#/192.168.27.1",
-	"dhcp_range": "192.168.27.100,192.168.27.150,1h",
-    "dhcp_option": "6,10.2.3.1",
-	"vendor_class": "set:device,IoT"
-    },
-    "host_apd_cfg": {
-	"ip": "192.168.27.1",
-	"ssid": "mistborn",
-	"wpa_passphrase":"iotwifipass",
-	"channel": "6"
-    },
-    "wpa_supplicant_cfg": {
-	"cfg_file": "/etc/wpa_supplicant/wpa_supplicant.conf"
-    }
-}

scripts/services/Mistborn-base.service

@@ -12,7 +12,7 @@ PermissionsStartOnly=true
 EnvironmentFile=/opt/mistborn/.env
 ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/base.yml down
 
-ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/base.yml build
+#ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/base.yml build
 ExecStartPre=-/sbin/ip address add 10.2.3.1/30 dev DIFACE
 ExecStartPre=/sbin/iptables -w -I DOCKER-USER -i DIFACE -p udp --dport 53 -j MISTBORN_LOG_DROP
 ExecStartPre=/sbin/iptables -w -I DOCKER-USER -i DIFACE -p tcp --dport 53 -j MISTBORN_LOG_DROP

scripts/services/Mistborn-mattermost.service

@@ -0,0 +1,21 @@
+[Unit]
+Description=Mistborn Mattermost Service
+Requires=Mistborn-base.service
+After=Mistborn-base.service
+
+[Service]
+Restart=always
+User=root
+Group=docker
+PermissionsStartOnly=true
+# Shutdown container (if running) when unit is stopped
+ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/extra/mattermost.yml down
+
+# Start container when unit is started
+ExecStart=/usr/local/bin/docker-compose --env-file /opt/mistborn/.envs/.production/.mattermost -f /opt/mistborn/extra/mattermost.yml up --build
+# Stop container when unit is stopped
+ExecStop=/usr/local/bin/docker-compose -f /opt/mistborn/extra/mattermost.yml down
+# Post stop
+
+[Install]
+WantedBy=multi-user.target

scripts/services/Mistborn-wifi.service

@@ -1,28 +0,0 @@
-[Unit]
-Description=Mistborn Wifi Service 
-Requires=Mistborn-base.service
-After=Mistborn-base.service
-
-[Service]
-Restart=always
-User=root
-Group=docker
-PermissionsStartOnly=true
-# Shutdown container (if running) when unit is stopped
-EnvironmentFile=/opt/mistborn/.env
-
-ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/extra/wifi.yml down
-ExecStartPre=/opt/mistborn/scripts/services/wifi/init.sh
-ExecStartPre=/usr/local/bin/docker-compose -f /opt/mistborn/extra/wifi.yml build
-# Start container when unit is started
-ExecStart=/usr/local/bin/docker-compose -f /opt/mistborn/extra/wifi.yml up
-
-ExecStartPost=/opt/mistborn/scripts/services/wifi/start.sh
-
-# Stop container when unit is stopped
-ExecStop=/usr/local/bin/docker-compose -f /opt/mistborn/extra/wifi.yml down
-# Post stop
-ExecStopPost=/opt/mistborn/scripts/services/wifi/stop.sh
-
-[Install]
-WantedBy=multi-user.target

scripts/services/wifi/init.sh

@@ -1,75 +0,0 @@
-#!/bin/bash
-
-# Colors
-MAGENTA='\e[0;35m'
-RED='\e[0;31m'
-GREEN='\e[0;32m'
-BLUE='\e[0;34m'
-NC='\e[0m'
-
-# Check that the interface exists and its not in use. Returns iface phy
-iface_check (){
-    IFACE="$1"
-    
-    # Check that the requested iface is available
-    if ! [ -e /sys/class/net/"$IFACE" ]
-    then
-        echo -e "${RED}[ERROR]${NC} The interface provided does not exist. Exiting..."
-        exit 1
-    fi
-   
-    # Check that the given interface is not used by the host as the default route
-    if [[ $(ip r | grep default | cut -d " " -f5) == "$IFACE" ]]
-    then
-        echo -e "${BLUE}[INFO]${NC} The selected interface is configured as the default route, if you use it you will lose internet connectivity"
-        exit 1;
-	fi
-
-    # Find the physical interface for the given wireless interface
-    PHY=$(cat /sys/class/net/"$IFACE"/phy80211/name)
-    echo $PHY
-}
-
-hostapd_setup() {
-    ### Check if hostapd is running in the host
-    hostapd_pid=$(pgrep hostapd)
-    if [ ! "$hostapd_pid" == "" ] 
-    then
-       echo -e "${BLUE}[INFO]${NC} hostapd service is already running in the system, make sure you use a different wireless interface..."
-       #kill -9 "$hostapd_pid"
-    fi
-
-    # Unblock wifi and bring the wireless interface up
-    echo -e "${BLUE}[INFO]${NC} Unblocking wifi and setting ${IFACE} up"
-    rfkill unblock wifi
-    ip link set "$IFACE" up
-
-    # Check if a wlan config file exists, else take wlan parameters by default
-    if [ -e "$PATHSCRIPT"/"$CONF_FILE" ]
-    then
-        echo -e "${BLUE}[INFO]${NC} Found WLAN config file"
-	    # Parse the wlan config file
-		IFS="="
-		while read -r name value
-		do
-                    case $name in
-                        ''|\#* ) continue;; # Skip blank lines and lines starting with #
-                        "SSID" )
-                            SSID=${value//\"/}
-                            echo -e "${BLUE}"[INFO]"${NC}" SSID: "${MAGENTA}""$SSID""${NC}";;
-                        "PASSPHRASE" )
-                            PASSPHRASE=${value//\"/};;
-                        "HW_MODE" )
-                            HW_MODE=${value//\"/};;
-                        "CHANNEL" )
-                            CHANNEL=${value//\"/};;
-                        * )
-                            echo Parameter "$name" in "$PATHSCRIPT"/"$CONF_FILE" not recognized
-		    esac
-		done < "$PATHSCRIPT"/"$CONF_FILE"
-    else
-        echo -e "${BLUE}[INFO]${NC} WLAN config file not found. Setting default WLAN parameters"
-        echo -e "${BLUE}"[INFO]"${NC}" SSID: "${MAGENTA}""$SSID""${NC}"
-    fi
-
-}

scripts/subinstallers/gen_prod_env.sh

@@ -72,3 +72,17 @@ echo "SIGNUPS_ALLOWED=true" >> $BITWARDEN_PROD_FILE
 JITSI_PROD_FILE="./.envs/.production/.jitsi"
 cp ./scripts/conf/jitsi.env $JITSI_PROD_FILE
 mkdir -p ./.envs/.production/.jitsi-cfg/{web/letsencrypt,transcripts,prosody,jicofo,jvb}
+sed -i "s/JICOFO_COMPONENT_SECRET.*/JICOFO_COMPONENT_SECRET=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")/" "$JITSI_PROD_FILE"
+sed -i "s/JICOFO_AUTH_PASSWORD.*/JICOFO_AUTH_PASSWORD=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")/" "$JITSI_PROD_FILE"
+sed -i "s/JVB_AUTH_PASSWORD.*/JVB_AUTH_PASSWORD=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")/" "$JITSI_PROD_FILE"
+sed -i "s/JIGASI_XMPP_PASSWORD.*/JIGASI_XMPP_PASSWORD=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")/" "$JITSI_PROD_FILE"
+sed -i "s/JIBRI_RECORDER_PASSWORD.*/JIBRI_RECORDER_PASSWORD=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")/" "$JITSI_PROD_FILE"
+sed -i "s/JIBRI_XMPP_PASSWORD.*/JIBRI_XMPP_PASSWORD=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")/" "$JITSI_PROD_FILE"
+
+# Mattermost
+MATTERMOST_PROD_FILE="./.envs/.production/.mattermost"
+echo "MM_USERNAME=mmuser" > $MATTERMOST_PROD_FILE
+echo "MM_PASSWORD=$1" >> $MATTERMOST_PROD_FILE
+echo "MM_DBNAME=mattermost" >> $MATTERMOST_PROD_FILE
+POSTGRES_PASSWORD=$(python3 -c "import secrets; import string; print(f''.join([secrets.choice(string.ascii_letters+string.digits) for x in range(32)]))")
+echo "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" >> $MATTERMOST_PROD_FILE

scripts/subinstallers/wifi.sh

@@ -1,3 +0,0 @@
-#!/bin/bash
-
-sudo apt-get install -y bridge-utils iw rfkill