Remove device_one_time_keys_count from is_empty() sync checks, fixing sync issue as reported by Nekron
Closes#115
See merge request famedly/conduit!215
gcc-8-aarch64-linux-gnu is not available in debian 11 (which the rust image uses), so update to gcc-10
Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>
- Build release builds for branches "master" and "next"
- Push docker images under different tags, depending on why the pipeline started
- branch master: push to `latest`
- branch next: push to `next`
- tag: push to `$TAG_NAME`
Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>
Using `$CI_COMMIT_REF_NAME` means we get `master` for every image build,
which is not very useful/informative. Using `$CI_COMMIT_SHORT_SHA`,
on the other hand, makes it possible to see exactly from which commit an
image was built.
The average german man has a life expectancy of 78.7 years, or 689884.2 hours.
Assuming that Timo is 20 years old, he has rougly 514564.2 hours left on planet earth.
Also assuming that cross release builds took him 25 minutes before,
but 2-2.5x of that with the current release compilation config he wasted roughly an hour waiting for it to complete.
If he continued to work on Conduit for 20 more years (or 175320 hours),
and makes a release compilation about once per day, this means 7305 hours or 304 days wasted waiting for the rust compiler.
By cutting that back down to the original settings, he get's 182 days of his life back.
That's about 0.63% of his remaining life.
182 joyful days he can spend with family and loved ones.
1. The fallback text of the get_pdu admin room command response message
now contains the same text as the formatted_body content (namely, the
json instead of Debug-formatting of a serde type).
2. The formatted_body content of the get_pdu response is now
html-escaped.
The spec does not require servers to apply a room canonical alias event upon room creation (yet).
Still, synapse does that, since users can set their desired alias in Elements room creation dialog.
With this commit, conduit also sets that alias if it is a valid one.
This closes https://gitlab.com/famedly/conduit/-/issues/123
4 years ago
74 changed files with 6494 additions and 3717 deletions
If you run into any problems while setting up an Appservice, write an email to `timo@koesters.xyz`, ask us in `#conduit:matrix.org` or [open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new).
If you run into any problems while setting up an Appservice, write an email to `timo@koesters.xyz`, ask us in `#conduit:matrix.org` or [open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new).
## Tested appservices
## Set up the appservice - general instructions
Here are some appservices we tested and that work with Conduit:
- matrix-appservice-discord
- mautrix-hangouts
- mautrix-telegram
## Set up the appservice
Follow whatever instructions are given by the appservice. This usually includes
Follow whatever instructions are given by the appservice. This usually includes
downloading, changing its config (setting domain, homeserver url, port etc.)
downloading, changing its config (setting domain, homeserver url, port etc.)
@ -46,3 +39,48 @@ Then you are done. Conduit will send messages to the appservices and the
appservice can send requests to the homeserver. You don't need to restart
appservice can send requests to the homeserver. You don't need to restart
Conduit, but if it doesn't work, restarting while the appservice is running
Conduit, but if it doesn't work, restarting while the appservice is running
could help.
could help.
## Appservice-specific instructions
### Tested appservices
These appservices have been tested and work with Conduit without any extra steps:
--- levels.users[self.main_intent.mxid] = 9001 if is_initial else 100
+++ levels.users[self.main_intent.mxid] = 100 if is_initial else 100
```
- [Between lines 1041 and 1042](https://github.com/mautrix/signal/blob/4ea831536f154aba6419d13292479eb383ea3308/mautrix_signal/portal.py#L1041-L1042) add a new line:
```diff
"type": str(EventType.ROOM_POWER_LEVELS),
+++ "state_key": "",
"content": power_levels.serialize(),
```
3. **Deploy the patch**. This is different depending on how you have `mautrix-signal` deployed:
- [*If using virtualenv*] Copy your patched `portal.py` to `./lib/python3.7/site-packages/mautrix_signal/portal.py` (the exact version of Python may be different on your system).
- [*If using Docker*] Map the patched `portal.py` into the `mautrix-signal` container:
If you run into any problems while setting up Conduit, write an email to `timo@koesters.xyz`, ask us in `#conduit:matrix.org` or [open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new).
If you run into any problems while setting up Conduit, write an email to `timo@koesters.xyz`, ask us
in `#conduit:matrix.org` or [open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new).
## Installing Conduit
## Installing Conduit
You may simply download the binary that fits your machine. Run `uname -m` to see
Although you might be able to compile Conduit for Windows, we do recommend running it on a linux server. We therefore
what you need. Now copy the right url:
only offer Linux binaries.
| CPU Architecture | GNU (Ubuntu, Debian, ArchLinux, ...) | MUSL (Alpine, ... ) |
You may simply download the binary that fits your machine. Run `uname -m` to see what you need. Now copy the right url:
Alternatively, you may compile the binary yourself using
Alternatively, you may compile the binary yourself using
```bash
```bash
$ cargo build --release
$ cargo build --release
```
```
Note that this currently requires Rust 1.50.
Note that this currently requires Rust 1.50.
If you want to cross compile Conduit to another architecture, read the [Cross-Compile Guide](CROSS_COMPILE.md).
If you want to cross compile Conduit to another architecture, read the [Cross-Compile Guide](CROSS_COMPILE.md).
## Adding a Conduit user
## Adding a Conduit user
While Conduit can run as any user it is usually better to use dedicated users for different services.
While Conduit can run as any user it is usually better to use dedicated users for different services. This also allows
This also allows you to make sure that the file permissions are correctly set up.
you to make sure that the file permissions are correctly set up.
In Debian you can use this command to create a Conduit user:
In Debian you can use this command to create a Conduit user:
```
```bash
sudo adduser --system conduit --no-create-home
sudo adduser --system conduit --no-create-home
```
```
## Setting up a systemd service
## Setting up a systemd service
Now we'll set up a systemd service for Conduit, so it's easy to start/stop
Now we'll set up a systemd service for Conduit, so it's easy to start/stop Conduit and set it to autostart when your
Conduit and set it to autostart when your server reboots. Simply paste the
server reboots. Simply paste the default systemd service you can find below into
default systemd service you can find below into
`/etc/systemd/system/conduit.service`.
`/etc/systemd/system/conduit.service`.
```systemd
```systemd
@ -74,10 +78,10 @@ Finally, run
$ sudo systemctl daemon-reload
$ sudo systemctl daemon-reload
```
```
## Creating the Conduit configuration file
## Creating the Conduit configuration file
Now we need to create the Conduit's config file in `/etc/matrix-conduit/conduit.toml`. Paste this in **and take a moment to read it. You need to change at least the server name.**
Now we need to create the Conduit's config file in `/etc/matrix-conduit/conduit.toml`. Paste this in **and take a moment
to read it. You need to change at least the server name.**
```toml
```toml
[global]
[global]
@ -104,8 +108,8 @@ port = 6167
# Max size for uploads
# Max size for uploads
max_request_size = 20_000_000 # in bytes
max_request_size = 20_000_000 # in bytes
# Disabling registration means no new users will be able to register on this server
# Enables registration. If set to false, no users can register on this server.
allow_registration = false
allow_registration = true
# Disable encryption, so no new encrypted rooms can be created
# Disable encryption, so no new encrypted rooms can be created
# Note: existing rooms will continue to work
# Note: existing rooms will continue to work
@ -125,21 +129,20 @@ address = "127.0.0.1" # This makes sure Conduit can only be reached using the re
## Setting the correct file permissions
## Setting the correct file permissions
As we are using a Conduit specific user we need to allow it to read the config.
As we are using a Conduit specific user we need to allow it to read the config. To do that you can run this command on
To do that you can run this command on Debian:
Debian:
```
```bash
sudo chown -R conduit:nogroup /etc/matrix-conduit
sudo chown -R conduit:nogroup /etc/matrix-conduit
```
```
If you use the default database path you also need to run this:
If you use the default database path you also need to run this:
Download or compile a Conduit binary, set up the config and call it from somewhere like a systemd script. [Read
Docker: [docker/README.md](docker/README.md)
more](DEPLOY.md)
If you want to connect an Appservice to Conduit, take a look at the [Appservice Guide](APPSERVICES.md).
##### Deploy using a Debian package
You need to have the `deb` helper command installed that creates Debian packages from Cargo projects (see [cargo-deb](https://github.com/mmstick/cargo-deb/) for more info):
```shell
$ cargo install cargo-deb
```
Then, you can create and install a Debian package at a whim:
> **Note:** To run and use Conduit you should probably use it with a Domain or Subdomain behind a reverse proxy (like Nginx, Traefik, Apache, ...) with a Lets Encrypt certificate.
> **Note:** To run and use Conduit you should probably use it with a Domain or Subdomain behind a reverse proxy (like Nginx, Traefik, Apache, ...) with a Lets Encrypt certificate.
## Docker
## Docker
### Build & Dockerfile
### Build & Dockerfile
The Dockerfile provided by Conduit has two stages, each of which creates an image.
The Dockerfile provided by Conduit has two stages, each of which creates an image.
1. **Builder:** Builds the binary from local context or by cloning a git revision from the official repository.
2. **Runtime:** Copies the built binary from **Builder** and sets up the runtime environment, like creating a volume to persist the database and applying the correct permissions.
The Dockerfile includes a few build arguments that should be supplied when building it.
``` Dockerfile
ARG LOCAL=false
ARG CREATED
ARG VERSION
ARG GIT_REF=origin/master
```
- **CREATED:** Date and time as string (date-time as defined by RFC 3339). Will be used to create the Open Container Initiative compliant label `org.opencontainers.image.created`. Supply by it like this `$(date -u +'%Y-%m-%dT%H:%M:%SZ')`
1. **Builder:** Builds the binary from local context or by cloning a git revision from the official repository.
- **VERSION:** The SemVer version of Conduit, which is in the image. Will be used to create the Open Container Initiative compliant label `org.opencontainers.image.version`. If you have a `Cargo.toml` in your build context, you can get it with `$(grep -m1 -o '[0-9].[0-9].[0-9]' Cargo.toml)`
2. **Runner:** Copies the built binary from **Builder** and sets up the runtime environment, like creating a volume to persist the database and applying the correct permissions.
- **LOCAL:***(Optional)* A boolean value, specifies if the local build context should be used, or if the official repository will be cloned. If not supplied with the build command, it will default to `false`.
- **GIT_REF:***(Optional)* A git ref, like `HEAD` or a commit ID. The supplied ref will be used to create the Open Container Initiative compliant label `org.opencontainers.image.revision` and will be the ref that is cloned from the repository when not building from the local context. If not supplied with the build command, it will default to `origin/master`.
To build the image you can use the following command
To build the image you can use the following command
which also will tag the resulting image as `matrixconduit/matrix-conduit:latest`.
which also will tag the resulting image as `matrixconduit/matrix-conduit:latest`.
**Note:** it ommits the two optional `build-arg`s.
### Run
### Run
After building the image you can simply run it with
After building the image you can simply run it with
```bash
```bash
docker run -d -p 8448:8000 -v ~/conduit.toml:/srv/conduit/conduit.toml -v db:/srv/conduit/.local/share/conduit matrixconduit/matrix-conduit:latest
docker run -d -p 8448:6167 -v ~/conduit.toml:/srv/conduit/conduit.toml -v db:/srv/conduit/.local/share/conduit matrixconduit/matrix-conduit:latest
```
```
For detached mode, you also need to use the `-d` flag. You also need to supply a `conduit.toml` config file, you can find an example [here](../conduit-example.toml).
or you can skip the build step and pull the image from one of the following registries:
The `-d` flag lets the container run in detached mode. You now need to supply a `conduit.toml` config file, an example can be found [here](../conduit-example.toml).
You can pass in different env vars to change config values on the fly. You can even configure Conduit completely by using env vars, but for that you need
You can pass in different env vars to change config values on the fly. You can even configure Conduit completely by using env vars, but for that you need
too pass `-e CONDUIT_CONFIG=""` into your container. For an overview of possible values, please take a look at the `docker-compose.yml` file.
to pass `-e CONDUIT_CONFIG=""` into your container. For an overview of possible values, please take a look at the `docker-compose.yml` file.
If you just want to test Conduit for a short time, you can use the `--rm` flag, which will clean up everything related to your container after you stop it.
If you just want to test Conduit for a short time, you can use the `--rm` flag, which will clean up everything related to your container after you stop it.
## Docker-compose
## Docker-compose
If the docker command is not for you or your setup, you can also use one of the provided `docker-compose` files. Depending on your proxy setup, use the [`docker-compose.traefik.yml`](docker-compose.traefik.yml) including [`docker-compose.override.traefik.yml`](docker-compose.override.traefik.yml) or the normal [`docker-compose.yml`](../docker-compose.yml) for every other reverse proxy.
If the docker command is not for you or your setup, you can also use one of the provided `docker-compose` files. Depending on your proxy setup, use the [`docker-compose.traefik.yml`](docker-compose.traefik.yml) and [`docker-compose.override.traefik.yml`](docker-compose.override.traefik.yml) for Traefik (don't forget to remove `.traefik` from the filenames) or the normal [`docker-compose.yml`](../docker-compose.yml) for every other reverse proxy. Additional info about deploying
Conduit can be found [here](../DEPLOY.md).
### Build
### Build
To build the Conduit image with docker-compose, you first need to open and modify the `docker-compose.yml` file. There you need to comment the `image:` option and uncomment the `build:` option. Then call docker-compose with:
To build the Conduit image with docker-compose, you first need to open and modify the `docker-compose.yml` file. There you need to comment the `image:` option and uncomment the `build:` option. Then call docker-compose with:
```bash
```bash
CREATED=$(date -u +'%Y-%m-%dT%H:%M:%SZ') VERSION=$(grep -m1 -o '[0-9].[0-9].[0-9]' Cargo.toml) docker-compose up
docker-compose up
```
```
This will also start the container right afterwards, so if want it to run in detached mode, you also should use the `-d` flag. For possible `build-args`, please take a look at the above `Build & Dockerfile` section.
This will also start the container right afterwards, so if want it to run in detached mode, you also should use the `-d` flag.
### Run
### Run
If you already have built the image, you can just start the container and everything else in the compose file in detached mode with:
If you already have built the image or want to use one from the registries, you can just start the container and everything else in the compose file in detached mode with:
```bash
```bash
docker-compose up -d
docker-compose up -d
```
```
> **Note:** Don't forget to modify and adjust the compose file to your needs.
### Use Traefik as Proxy
As a container user, you probably know about Traefik. It is a easy to use reverse proxy for making containerized app and services available through the web. With the
two provided files, [`docker-compose.traefik.yml`](docker-compose.traefik.yml) and [`docker-compose.override.traefik.yml`](docker-compose.override.traefik.yml), it is
equally easy to deploy and use Conduit, with a little caveat. If you already took a look at the files, then you should have seen the `well-known` service, and that is
the little caveat. Traefik is simply a proxy and loadbalancer and is not able to serve any kind of content, but for Conduit to federate, we need to either expose ports
`443` and `8448` or serve two endpoints `.well-known/matrix/client` and `.well-known/matrix/server`.
With the service `well-known` we use a single `nginx` container that will serve those two files.
So...step by step:
1. Copy [`docker-compose.traefik.yml`](docker-compose.traefik.yml) and [`docker-compose.override.traefik.yml`](docker-compose.override.traefik.yml) from the repository and remove `.traefik` from the filenames.
2. Open both files and modify/adjust them to your needs. Meaning, change the `CONDUIT_SERVER_NAME` and the volume host mappings according to your needs.
3. Create the `conduit.toml` config file, an example can be found [here](../conduit-example.toml), or set `CONDUIT_CONFIG=""` and configure Conduit per env vars.
4. Uncomment the `element-web` service if you want to host your own Element Web Client and create a `element_config.json`.
5. Create the files needed by the `well-known` service.
- `./nginx/matrix.conf` (relative to the compose file, you can change this, but then also need to change the volume mapping)
```nginx
server {
server_name <SUBDOMAIN>.<DOMAIN>;
listen 80 default_server;
location /.well-known/matrix/ {
root /var/www;
default_type application/json;
add_header Access-Control-Allow-Origin *;
}
}
```
- `./nginx/www/.well-known/matrix/client` (relative to the compose file, you can change this, but then also need to change the volume mapping)
```json
{
"m.homeserver": {
"base_url": "https://<SUBDOMAIN>.<DOMAIN>"
}
}
```
- `./nginx/www/.well-known/matrix/server` (relative to the compose file, you can change this, but then also need to change the volume mapping)
```json
{
"m.server": "<SUBDOMAIN>.<DOMAIN>:443"
}
```
6. Run `docker-compose up -d`
7. Connect to your homeserver with your preferred client and create a user. You should do this immediatly after starting Conduit, because the first created user is the admin.
"Thanks for trying out Conduit! This software is still in development, so expect many bugs and missing features. If you have federation enabled, you can join the Conduit chat room by typing `/join #conduit:matrix.org`. **Important: Please don't join any other Matrix rooms over federation without permission from the room's admins.** Some actions might trigger bugs in other server implementations, breaking the chat for everyone else.".to_owned(),
"## Thank you for trying out Conduit!\n\nConduit is currently in Beta. This means you can join and participate in most Matrix rooms, but not all features are supported and you might run into bugs from time to time.\n\nHelpful links:\n> Website: https://conduit.rs\n> Git and Documentation: https://gitlab.com/famedly/conduit\n> Report issues: https://gitlab.com/famedly/conduit/-/issues\n\nHere are some rooms you can join (by typing the command):\n\nConduit room (Ask questions and get notified on updates):\n`/join #conduit:fachschaften.org`\n\nConduit lounge (Off-topic, only Conduit users are allowed to join)\n`/join #conduit-lounge:conduit.rs`".to_owned(),
"Thanks for trying out Conduit! This software is still in development, so expect many bugs and missing features. If you have federation enabled, you can join the Conduit chat room by typing <code>/join #conduit:matrix.org</code>. <strong>Important: Please don't join any other Matrix rooms over federation without permission from the room's admins.</strong> Some actions might trigger bugs in other server implementations, breaking the chat for everyone else.".to_owned(),
"<h2>Thank you for trying out Conduit!</h2>\n<p>Conduit is currently in Beta. This means you can join and participate in most Matrix rooms, but not all features are supported and you might run into bugs from time to time.</p>\n<p>Helpful links:</p>\n<blockquote>\n<p>Website: https://conduit.rs<br>Git and Documentation: https://gitlab.com/famedly/conduit<br>Report issues: https://gitlab.com/famedly/conduit/-/issues</p>\n</blockquote>\n<p>Here are some rooms you can join (by typing the command):</p>\n<p>Conduit room (Ask questions and get notified on updates):<br><code>/join #conduit:fachschaften.org</code></p>\n<p>Conduit lounge (Off-topic, only Conduit users are allowed to join)<br><code>/join #conduit-lounge:conduit.rs</code></p>\n".to_owned(),
))
))
.expect("event is valid, we just created it"),
.expect("event is valid, we just created it"),
unsigned: None,
unsigned: None,
@ -524,9 +536,16 @@ pub async fn register_route(
///
///
/// Changes the password of this account.
/// Changes the password of this account.
///
///
/// - Invalidates all other access tokens if logout_devices is true
/// - Requires UIAA to verify user password
/// - Deletes all other devices and most of their data (to-device events, last seen, etc.) if
/// - Changes the password of the sender user
/// logout_devices is true
/// - The password hash is calculated using argon2 with 32 character salt, the plain password is
/// not saved
///
/// If logout_devices is true it does the following for each device except the sender device:
/// - Invalidates access token
/// - Deletes device metadata (device id, device display name, last seen ip, last seen ts)
.map_err(|_|Error::BadConfig("Database folder doesn't exists and couldn't be created (e.g. due to missing permissions). Please create the database folder yourself."))?;
}
letbuilder=Engine::open(&config)?;
letbuilder=Engine::open(config)?;
ifconfig.max_request_size<1024{
ifconfig.max_request_size<1024{
eprintln!("ERROR: Max request size is less than 1KB. Please increase it.");
eprintln!("ERROR: Max request size is less than 1KB. Please increase it.");
.map_err(|_|Error::bad_database("Invalid edu event in read_receipts."))?;
.map_err(|_|Error::bad_database("Invalid edu event in read_receipts."))?;
letfederation_event=matchevent{
letfederation_event=matchevent{
AnySyncEphemeralRoomEvent::Receipt(r)=>{
AnySyncEphemeralRoomEvent::Receipt(r)=>{
@ -378,6 +392,22 @@ impl Sending {
}
}
}
}
foruser_idindevice_list_changes{
// Empty prev id forces synapse to resync: https://github.com/matrix-org/synapse/blob/98aec1cc9da2bd6b8e34ffb282c85abf9b8b42ca/synapse/handlers/device.py#L767
// Because synapse resyncs, we can just insert dummy data
Timo Kösters
Jonas Platte
Jonas Zohren
Moritz Bitsch
Jonas Platte
Nyaaori
Andrew Morgan
Luc-pascal Ceccaldi
Greg Sutcliffe
Jonas Zohren
Daniel Wiesenberg
Jonathan de Jong
Rasmus Thomsen
charludo
Kurt Roeckx
Devin Ragotzy
Tom Smeding