docs: WIP: Adjust suggested value ranges to Timo's suggestion and when to change them

Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>
docs: WIP: Added some suggested range for numeric values
2021-09-15 12:45:13 +02:00 · 2021-09-15 12:38:17 +02:00 · 2021-09-15 12:21:43 +02:00 · 2021-09-15 11:16:19 +02:00 · 2021-09-15 11:16:19 +02:00 · 2021-09-15 11:16:18 +02:00
36 changed files with 1198 additions and 1127 deletions
@@ -3,6 +3,7 @@ stages:
  - build docker image
  - test
  - upload artifacts
+  - publish

 variables:
  GIT_SUBMODULE_STRATEGY: recursive
@@ -75,12 +76,12 @@ build:release:cargo:aarch64-unknown-linux-gnu:
  extends: .build-cargo-shared-settings
  variables:
    TARGET: "aarch64-unknown-linux-gnu"
-    NEEDED_PACKAGES: "build-essential gcc-10-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-dev-arm64-cross"
+    NEEDED_PACKAGES: "build-essential gcc-8-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-dev-arm64-cross"
    CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
    CC_aarch64_unknown_linux_gnu: aarch64-linux-gnu-gcc
    CXX_aarch64_unknown_linux_gnu: aarch64-linux-gnu-g++
-    TARGET_CC: "/usr/bin/aarch64-linux-gnu-gcc-10"
-    TARGET_AR: "/usr/bin/aarch64-linux-gnu-gcc-ar-10"
+    TARGET_CC: "/usr/bin/aarch64-linux-gnu-gcc-8"
+    TARGET_AR: "/usr/bin/aarch64-linux-gnu-gcc-ar-8"
  artifacts:
    name: "conduit-aarch64-unknown-linux-gnu"
    paths:
@@ -222,7 +223,6 @@ build:docker:next:
    - >
      /kaniko/executor
      $KANIKO_CACHE_ARGS
-      --force
      --context $CI_PROJECT_DIR
      --build-arg CREATED=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
      --build-arg VERSION=$(grep -m1 -o '[0-9].[0-9].[0-9]' Cargo.toml)
@@ -370,3 +370,17 @@ publish:package:
    - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file conduit-x86_64-unknown-linux-gnu.deb "${BASE_URL}/conduit-x86_64-unknown-linux-gnu.deb"'


+pages:
+  stage: "publish"
+  image:
+    name: "squidfunk/mkdocs-material"
+    entrypoint: [""]
+  tags: ["docker"]
+  needs: []
+  rules:
+    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
+  script:
+    - "mkdocs build --site-dir public"
+  artifacts:
+    paths:
+      - "public"
@@ -1,86 +0,0 @@
-# Setting up Appservices
-
-## Getting help
-
-If you run into any problems while setting up an Appservice, write an email to `timo@koesters.xyz`, ask us in `#conduit:matrix.org` or [open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new).
-
-## Set up the appservice - general instructions
-
-Follow whatever instructions are given by the appservice. This usually includes
-downloading, changing its config (setting domain, homeserver url, port etc.)
-and later starting it.
-
-At some point the appservice guide should ask you to add a registration yaml
-file to the homeserver. In Synapse you would do this by adding the path to the
-homeserver.yaml, but in Conduit you can do this from within Matrix:
-
-First, go into the #admins room of your homeserver. The first person that
-registered on the homeserver automatically joins it. Then send a message into
-the room like this:
-
-    @conduit:your.server.name: register_appservice
-    ```
-    paste
-    the
-    contents
-    of
-    the
-    yaml
-    registration
-    here
-    ```
-
-You can confirm it worked by sending a message like this:
-`@conduit:your.server.name: list_appservices`
-
-The @conduit bot should answer with `Appservices (1): your-bridge`
-
-Then you are done. Conduit will send messages to the appservices and the
-appservice can send requests to the homeserver. You don't need to restart
-Conduit, but if it doesn't work, restarting while the appservice is running
-could help.
-
-## Appservice-specific instructions
-
-### Tested appservices
-
-These appservices have been tested and work with Conduit without any extra steps:
-
- [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord)
- [mautrix-hangouts](https://github.com/mautrix/hangouts/)
- [mautrix-telegram](https://github.com/mautrix/telegram/)
-
-### [mautrix-signal](https://github.com/mautrix/signal)
-
-There are a few things you need to do, in order for the Signal bridge (at least
-up to version `0.2.0`) to work. How you do this depends on whether you use
-Docker or `virtualenv` to run it. In either case you need to modify
-[portal.py](https://github.com/mautrix/signal/blob/master/mautrix_signal/portal.py).
-Do this **before** following the bridge installation guide.
-
-1. **Create a copy of `portal.py`**. Go to
-   [portal.py](https://github.com/mautrix/signal/blob/master/mautrix_signal/portal.py)
-at [mautrix-signal](https://github.com/mautrix/signal) (make sure you change to
-the correct commit/version of mautrix-signal you're using) and copy its
-content. Create a new `portal.py` on your system and paste the content in.
-2. **Patch the copy**. Exact line numbers may be slightly different, look nearby if they don't match: 
-  - [Line 1020](https://github.com/mautrix/signal/blob/4ea831536f154aba6419d13292479eb383ea3308/mautrix_signal/portal.py#L1020)
-    ```diff
-    --- levels.users[self.main_intent.mxid] = 9001 if is_initial else 100
-    +++ levels.users[self.main_intent.mxid] = 100 if is_initial else 100
-    ```
-  - [Between lines 1041 and 1042](https://github.com/mautrix/signal/blob/4ea831536f154aba6419d13292479eb383ea3308/mautrix_signal/portal.py#L1041-L1042) add a new line:
-    ```diff
-        "type": str(EventType.ROOM_POWER_LEVELS),
-    +++ "state_key": "",
-        "content": power_levels.serialize(),
-    ```
-3. **Deploy the patch**. This is different depending on how you have `mautrix-signal` deployed:
-  - [*If using virtualenv*] Copy your patched `portal.py` to `./lib/python3.7/site-packages/mautrix_signal/portal.py` (the exact version of Python may be different on your system).
-  -  [*If using Docker*] Map the patched `portal.py` into the `mautrix-signal` container:
-    
-    ```yaml
-    volumes:
-      - ./your/path/on/host/portal.py:/usr/lib/python3.9/site-packages/mautrix_signal/portal.py
-    ```
-4. Now continue with the [bridge installation instructions ](https://docs.mau.fi/bridges/index.html) and the general bridge notes above.
@@ -1,11 +0,0 @@
-Install docker:
-
-```
-$ sudo apt install docker
-$ sudo usermod -aG docker $USER
-$ exec sudo su -l $USER
-$ sudo systemctl start docker
-$ cargo install cross
-$ cross build --release --target armv7-unknown-linux-musleabihf
-```
-The cross-compiled binary is at target/armv7-unknown-linux-musleabihf/release/conduit
@@ -254,7 +254,6 @@ dependencies = [
 "opentelemetry-jaeger",
 "parking_lot",
 "rand 0.8.4",
- "rayon",
 "regex",
 "reqwest",
 "ring",
@@ -1748,31 +1747,6 @@ dependencies = [
 "rand_core 0.6.3",
 ]

-[[package]]
-name = "rayon"
-version = "1.5.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c06aca804d41dbc8ba42dfd964f0d01334eceb64314b9ecf7c5fad5188a06d90"
-dependencies = [
- "autocfg",
- "crossbeam-deque",
- "either",
- "rayon-core",
-]
-
-[[package]]
-name = "rayon-core"
-version = "1.9.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d78120e2c850279833f1dd3582f730c4ab53ed95aeaaaa862a2a5c71b1656d8e"
-dependencies = [
- "crossbeam-channel",
- "crossbeam-deque",
- "crossbeam-utils 0.8.5",
- "lazy_static",
- "num_cpus",
-]
-
 [[package]]
 name = "redox_syscall"
 version = "0.2.10"
@@ -1994,7 +1968,7 @@ dependencies = [
 [[package]]
 name = "ruma"
 version = "0.4.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "assign",
 "js_int",
@@ -2015,7 +1989,7 @@ dependencies = [
 [[package]]
 name = "ruma-api"
 version = "0.18.3"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "bytes",
 "http",
@@ -2031,7 +2005,7 @@ dependencies = [
 [[package]]
 name = "ruma-api-macros"
 version = "0.18.3"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "proc-macro-crate",
 "proc-macro2",
@@ -2042,7 +2016,7 @@ dependencies = [
 [[package]]
 name = "ruma-appservice-api"
 version = "0.4.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "ruma-api",
 "ruma-common",
@@ -2056,7 +2030,7 @@ dependencies = [
 [[package]]
 name = "ruma-client-api"
 version = "0.12.2"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "assign",
 "bytes",
@@ -2076,7 +2050,7 @@ dependencies = [
 [[package]]
 name = "ruma-common"
 version = "0.6.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "indexmap",
 "js_int",
@@ -2091,7 +2065,7 @@ dependencies = [
 [[package]]
 name = "ruma-events"
 version = "0.24.5"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "indoc",
 "js_int",
@@ -2107,7 +2081,7 @@ dependencies = [
 [[package]]
 name = "ruma-events-macros"
 version = "0.24.5"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "proc-macro-crate",
 "proc-macro2",
@@ -2118,7 +2092,7 @@ dependencies = [
 [[package]]
 name = "ruma-federation-api"
 version = "0.3.1"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "js_int",
 "ruma-api",
@@ -2133,7 +2107,7 @@ dependencies = [
 [[package]]
 name = "ruma-identifiers"
 version = "0.20.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "paste",
 "rand 0.8.4",
@@ -2147,7 +2121,7 @@ dependencies = [
 [[package]]
 name = "ruma-identifiers-macros"
 version = "0.20.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "quote",
 "ruma-identifiers-validation",
@@ -2157,12 +2131,12 @@ dependencies = [
 [[package]]
 name = "ruma-identifiers-validation"
 version = "0.5.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"

 [[package]]
 name = "ruma-identity-service-api"
 version = "0.3.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "js_int",
 "ruma-api",
@@ -2175,7 +2149,7 @@ dependencies = [
 [[package]]
 name = "ruma-push-gateway-api"
 version = "0.3.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "js_int",
 "ruma-api",
@@ -2190,7 +2164,7 @@ dependencies = [
 [[package]]
 name = "ruma-serde"
 version = "0.5.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "bytes",
 "form_urlencoded",
@@ -2204,7 +2178,7 @@ dependencies = [
 [[package]]
 name = "ruma-serde-macros"
 version = "0.5.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "proc-macro-crate",
 "proc-macro2",
@@ -2215,7 +2189,7 @@ dependencies = [
 [[package]]
 name = "ruma-signatures"
 version = "0.9.0"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "base64 0.13.0",
 "ed25519-dalek",
@@ -2232,7 +2206,7 @@ dependencies = [
 [[package]]
 name = "ruma-state-res"
 version = "0.4.1"
-source = "git+https://github.com/timokoesters/ruma?rev=6e9823de284967a41100db0e3134319f4b6a0cfa#6e9823de284967a41100db0e3134319f4b6a0cfa"
+source = "git+https://github.com/ruma/ruma?rev=a6a1224652912a957b09f136ec5da2686be6e0e2#a6a1224652912a957b09f136ec5da2686be6e0e2"
 dependencies = [
 "itertools 0.10.1",
 "js_int",
@@ -19,8 +19,8 @@ rocket = { version = "0.5.0-rc.1", features = ["tls"] } # Used to handle request

 # Used for matrix spec type definitions and helpers
 #ruma = { version = "0.4.0", features = ["compat", "rand", "appservice-api-c", "client-api", "federation-api", "push-gateway-api-c", "state-res", "unstable-pre-spec", "unstable-exhaustive-types"] }
-#ruma = { git = "https://github.com/ruma/ruma", rev = "a6a1224652912a957b09f136ec5da2686be6e0e2", features = ["compat", "rand", "appservice-api-c", "client-api", "federation-api", "push-gateway-api-c", "state-res", "unstable-pre-spec", "unstable-exhaustive-types"] }
-ruma = { git = "https://github.com/timokoesters/ruma", rev = "6e9823de284967a41100db0e3134319f4b6a0cfa", features = ["compat", "rand", "appservice-api-c", "client-api", "federation-api", "push-gateway-api-c", "state-res", "unstable-pre-spec", "unstable-exhaustive-types"] }
+ruma = { git = "https://github.com/ruma/ruma", rev = "a6a1224652912a957b09f136ec5da2686be6e0e2", features = ["compat", "rand", "appservice-api-c", "client-api", "federation-api", "push-gateway-api-c", "state-res", "unstable-pre-spec", "unstable-exhaustive-types"] }
+#ruma = { git = "https://github.com/timokoesters/ruma", rev = "50c1db7e0a3a21fc794b0cce3b64285a4c750c71", features = ["compat", "rand", "appservice-api-c", "client-api", "federation-api", "push-gateway-api-c", "state-res", "unstable-pre-spec", "unstable-exhaustive-types"] }
 #ruma = { path = "../ruma/crates/ruma", features = ["compat", "rand", "appservice-api-c", "client-api", "federation-api", "push-gateway-api-c", "state-res", "unstable-pre-spec", "unstable-exhaustive-types"] }

 # Used for long polling and federation sender, should be the same as rocket::tokio
@@ -79,7 +79,6 @@ num_cpus = "1.13.0"
 threadpool = "1.8.1"
 heed = { git = "https://github.com/timokoesters/heed.git", rev = "f6f825da7fb2c758867e05ad973ef800a6fe1d5d", optional = true }
 thread_local = "1.1.3"
-rayon = "1.5.1"

 [features]
 default = ["conduit_bin", "backend_sqlite"]
@@ -0,0 +1,16 @@
+[book]
+title = "Conduit Docs"
+author = "The Conduit contributors"
+description = "Conduit is a simple, fast and reliable chat server for the Matrix protocol"
+language = "en"
+src = "docs"
+
+[rust]
+edition = "2018"
+
+[build]
+build-dir = "public"
+create-missing = true
+
+[output.html.search]
+limit-results = 15
@@ -0,0 +1,16 @@
+# Summary
+
+[Home](index.md)
+
+- [Installation](installation.md)
+    - [Prerequisites](installation/prerequisites.md)
+    - [From Binaries](installation/manual.md)
+    - [From Packages](installation/packages.md)
+    - [With Docker](installation/docker.md)
+- [Configuration](configuration.md)
+    - [Config options](configuration/configuration.md)
+    - [Appservices](configuration/appservices.md)
+- [Contribute](contribute.md)
+    - [Basics](development/basics.md)
+    - [Cross compilation](development/cross-compilation.md)
+    - [Tests & CI](development/tests-ci.md)
@@ -0,0 +1 @@
+# Configuration
@@ -0,0 +1,2 @@
+# Configuring Conduit
+
@@ -0,0 +1,220 @@
+# Configuring Conduit
+
+Conduit can be configured via a config file (conventionally called Conduit.toml) or environment variables. If a config
+file exists and environment variables are set, environment variables overwrite config options.
+
+You absolutely need to set the environment variable `CONDUIT_CONFIG_FILE` to either point to a config file (
+e.g. `CONDUIT_CONFIG_FILE=/etc/conduit/Conduit.toml`) or to an empty string (`CONDUIT_CONFIG_FILE=''`) if you want to
+configure Conduit with just environment variables.
+
+## Mandatory config options
+
+Mandatory variables must be configured in order for Conduit to run properly.
+
+### Server Name
+
+- Config file key: `server_name`
+- Envirnoment variable: `CONDUIT_SERVER_NAME`
+- Default value: _None, you will need to choose your own._
+
+The server_name is the name of this server. It is used as a suffix for user and room ids. Example: If you set it
+to `conduit.rs`, your usernames will look like `@somebody:conduit.rs`.
+
+The Conduit server needs to be reachable at https://your.server.name/ on port 443 (client-server) and 8448 (
+server-server) OR you can create /.well-known files to redirect requests. See
+the [Client-Server specs](https://matrix.org/docs/spec/client_server/latest#get-well-known-matrix-client) and
+the [Server-Server specs](https://matrix.org/docs/spec/server_server/r0.1.4#get-well-known-matrix-server) for more
+information.
+
+### Database Path
+
+- Config file key: `database_path`
+- Envirnoment variable: `CONDUIT_DATABASE_PATH`
+- Default value: _None, but many people like to use `/var/lib/conduit/`_.
+
+A **directory** where Conduit stores its database and media files. This directory must exist, have enough free space and
+be readable and writable by the user Conduit is running as.
+
+What does _enough free space_ mean? It heavily on the amount of messages your Conduit server will see and the amount and
+size of media files users on your Conduit server send. As a rule of thumb, you should have at least 10 GB of free space
+left. You should be comfortable for quite some time with 50 GB.
+
+### TCP Port
+
+- Config file key: `port`
+- Environment variable: `CONDUIT_PORT`
+- Default value: _None, but many people like to use `6167`_.
+
+The TCP port Conduit will listen on for connections. The port needs to be free (no other program is listeing on it).
+
+Conduit does currently (2021-09) not offer HTTPS by itself. Only unencrypted HTTP requests will be accepted on this
+port. Unless you know what you are doing, this port should not be exposed to the internet. Instead, use a reverse proxy
+capable of doing TLS to offer your Conduit server to the internet via HTTPS. See [TODO] for example configurations.
+
+## Optional configuration options
+
+These config options come with defaults and don't need to be configured for Conduit to run. That said, you should still
+check them to make sure that your Conduit server behaves like you want it to do.
+
+### Maximum request size
+
+- Config file key: `max_request_size`
+- Environment variable: `CONDUIT_MAX_REQUEST_SIZE`
+- Default value: `20_000_000` (~= 20 MB)
+
+The maximum size in bytes for incoming requests to Conduit. You can use underscores to improve readability.
+
+This will effectively limit the size for images, videos and other files users on your Conduit server can send.
+
+### Allow Registration?
+
+- Config file key: `allow_registration`
+- Environment variable: `CONDUIT_ALLOW_REGISTRATION`
+- Default value: `true`
+- Possible values: `true`, `false`
+
+It this is set to `false`, no new users can register accounts on your Conduit server. Already registered users will not
+be affected from this setting and can continue to user your server.
+
+The first user to ever register on your Conduit server will be considered the admin account and is automatically invited
+into the admin room.
+
+### Allow Encryption?
+
+- Config file key: `allow_encryption`
+- Environment variable: `CONDUIT_ALLOW_ENCRYPTION`
+- Default value: `true`
+- Possible values: `true`, `false`
+
+If this is set to `false`, Conduit disables the ability for users to create encrypted chats. Existing encrypted chats
+may continue to work.
+
+### Allow federation?
+
+- Config file key: `allow_federation`
+- Environment variable: `CONDUIT_ALLOW_FEDERATION`
+- Default value: `false`
+- Possible values: `true`, `false`
+
+Federation means that users from different Matrix servers can chat with each other. E.g. `@mathew:matrix.org` can chat
+with `@timo:conduit.rs`.
+
+If this option is set to `false`, users on your Conduit server can only talk with other users on your Conduit server.
+
+Federation with other servers needs to happen over HTTPS, so make sure you have set up a reverse proxy.
+
+### Jaeger Tracing
+
+- Config file key: `allow_jaeger`
+- Environment variable: `CONDUIT_ALLOW_JAEGER`
+- Default value: `false`
+- Possible values: `true`, `false`
+
+Enable Jaeger to support monitoring and troubleshooting through Jaeger.
+
+If you don't know what Jaeger is, you can safely leave this set to `false`.
+
+### Trusted servers
+
+- Config file key: `trusted_servers`
+- Environment variable: `CONDUIT_TRUSTED_SERVERS`
+- Default value: `[]`
+- Possible values: JSON-Array of server domains, e.g. `["matrix.org"]` or `["matrix.org", "conduit.rs"]`.
+
+Matrix servers have so-called "server keys", which authenticate messages from their users. Because your Conduit server
+might not know the server keys from every server it encounters, it can ask a _trusted server_ for them. This speeds
+things up for rooms with people from a lot of different servers.
+
+You should only set this to include trustworthy servers. Most people consider a good default to be `["matrix.org"]`.
+
+Only relevant if you have federation enabled.
+
+### Limit amount of concurrent requests
+
+- Config file key: `max_concurrent_requests`
+- Environment variable: `CONDUIT_MAX_CONCURRENT_REQUESTS`
+- Default value: `100`
+- Suggested values: `1` - `1000` (u16)
+
+How many requests Conduit can make at the same time. This affects federation with other Matrix servers, push
+notifications and app_services.
+
+// TODO Timo: When does it make sense to change this?
+
+### Configure logging
+
+- Config file key: `log`
+- Environment variable: `CONDUIT_LOG`
+- Default value: `info,state_res=warn,rocket=off,_=off,sled=off`
+
+Configures which kind of messages Conduit logs.
+
+> // TODO: Better and more thorough explanation
+
+### Worker threads
+
+- Config file key: `workers`
+- Environment variable: `CONDUIT_WORKERS`
+- Default value: cpu core count * 2
+- Possible values: // TODO
+
+> // TODO: Which thing exactly threads? What not?
+
+### Listening address
+
+- Config file key: `address`
+- Environment variable: `CONDUIT_ADDRESS`
+- Default value: `127.0.0.1`
+- Possible values: Valid IP addresses.
+
+Which IP address conduit is listening on. 127.0.0.1 means that Conduit can only be accessed from the same server or
+through a reverse proxy on that server. If you want it to be accessible from any network interface (which you should
+not, because other matrix servers should talk to your Conduit via a reverse proxy and not directly), you can set it
+to `0.0.0.0`.
+
+### Database cache capacity
+
+- Config file key: `db_cache_capacity_mb`
+- Environment variable: `CONDUIT_DB_CACHE_CAPACITY_MB`
+- Default value: `200`
+- Possible values: `true`, `false`
+
+The total amount of memory (RAM) that the database cache will be able to use.
+
+> // TODO: this needs clearification: In RAM or on disk and for what exactly?
+
+### PDU cache capacity
+
+- Config file key: `pdu_cache_capacity`
+- Environment variable: `CONDUIT_PDU_CACHE_CAPACITY`
+- Default value: `100_000`
+- Suggested values: `1_000` - `1_000_000` (u32)
+
+The total capacity (read: number of items) the pdu cache can hold in memory. Setting this to a lower number may slow
+Conduit down, as it must fetch more data from the database. Increasing it will mean that Conduit will start to use more
+memory as the cache slowly fills up.
+
+### SQLite WAL clean interval
+
+- Config file key: `sqlite_wal_clean_second_interval`
+- Environment variable: `CONDUIT_SQLITE_WAL_CLEAN_SECOND_INTERVAL`
+- Default value: `60` (every 60 seconds)
+- Suggested values: `1` - `3600` (u32)
+
+How often the WAL file should be cleaned up. The WAL file will be written to until cleaned up, after which it restarts
+writing from the beginning.
+
+The file's size will correspond to how long it could write to it in one go. (e.g. if conduit writes 100MB of data to the
+database inbetween that period, the file will grow to 100MB). You can read more about that in
+the [SQLite Docs](https://www.sqlite.org/draft/wal.html).
+
+Reducing this down too much can offset the benefits of using a WAL at all. However, having this too high can result in a
+large WAL file.
+
+Only relevant when using SQLite as the database.
+
+### Still undocumented config options
+
+- `tracing_flame`
+- `proxy`
+- `jwt_secret`
@@ -0,0 +1 @@
+# Contribute
@@ -0,0 +1 @@
+# Basics
@@ -0,0 +1,13 @@
+# Cross compilation
+
+Install docker:
+
+```bash
+sudo apt install docker
+sudo usermod -aG docker $USER
+exec sudo su -l $USER
+sudo systemctl start docker
+cargo install cross
+cross build --release --target armv7-unknown-linux-musleabihf
+```
+The cross-compiled binary is at target/armv7-unknown-linux-musleabihf/release/conduit
@@ -0,0 +1 @@
+# Tests & CI
@@ -0,0 +1,52 @@
+# Conduit
+
+Conduit is a simple, fast and reliable chat server for the [Matrix] protocol written in [Rust].
+
+-----
+> Note: This project is work-in-progress. Do *not* rely on it yet.
+
+## What is Matrix?
+
+[Matrix] is an open network for secure and decentralized
+communication. It allows you to chat with friends even if they are using
+another servers and client. You can even use bridges to communicate with users
+outside of Matrix, like a community on Discord or your family on Hangouts.
+
+## Why Conduit?
+
+Conduit is an open-source server implementation of the [Matrix
+Specification] with a focus on easy setup and low
+system requirements, making it very easy to set up.
+
+Other server implementations try to be extremely scalable, which makes sense if
+the goal is to support millions of users on a single instance, but makes
+smaller deployments a lot more inefficient. Conduit tries to keep it simple but
+takes full advantage of that, for example by using an in-memory database for
+[huge performance gains](https://github.com/timokoesters/romeo-and-juliet-benchmark).
+
+The future for Conduit in peer-to-peer Matrix (every client contains a server)
+is also bright.
+
+Conduit tries to be reliable by using the Rust programming language and paying
+close attention to error handling to make sure that evil clients, misbehaving
+servers or even a partially broken database will not cause the whole server to
+stop working.
+
+## Chat with us!
+
+We have a room on Matrix: [#conduit:matrix.org](https://matrix.to/#/#conduit:matrix.org)
+
+You can also contact us using:
+- Matrix: [@timo:koesters.xyz](https://matrix.to/#/@timo:koesters.xyz)
+- Email: [conduit@koesters.xyz](mailto:conduit@koesters.xyz)
+
+
+## Donate
+
+Liberapay: <https://liberapay.com/timokoesters/>\
+Bitcoin: `bc1qnnykf986tw49ur7wx9rpw2tevpsztvar5x8w4n`
+
+
+[Matrix]: https://matrix.org/
+[Rust]: https://rust-lang.org
+[Matrix Specification]: https://matrix.org/docs/spec
@@ -0,0 +1 @@
+# Installation
@@ -0,0 +1,2 @@
+:::warning Test
+:::
@@ -0,0 +1,12 @@
+# Distribution packages
+
+## Debian / Ubuntu
+
+[@paul:luon.net](https://matrix.to/#/@paul:luon.net) plans to package Conduit for Debian as soon as it reaches 1.0.
+Until it is available in the official repos, you can install the development version of it manually:
+
+```bash
+sudo apt-get install ca-certificates
+wget --https-only -O /tmp/conduit.deb https://gitlab.com/famedly/conduit/-/jobs/artifacts/master/raw/conduit-x86_64-unknown-linux-gnu.deb?job=build:cargo-deb:x86_64-unknown-linux-gnu
+sudo dpkg -i /tmp/conduit.deb
+```
@@ -0,0 +1,31 @@
+# Prerequisites for running Conduit
+
+You'll need:
+
+- A domain. Commonly cost about $10/year.
+- A Linux server with a stable internet connection, at least 500 MB of RAM and some disk space for messages and
+  attachments. Commonly start at $5/month.
+- Some basic knowledge about using a shell, SSH and configuring and protecting a server.
+  
+  
+## A word of caution:
+
+Don't underestimate the toll of administrating your own server.
+Conduit can't protect your conversations if your server gets compromised or deleted.
+
+Make sure that you got:
+
+- Automatic security updates
+    - On Ubuntu/Debian: Set up unattended-upgrades
+    - On RHEL/CentOS: Have a look at yum-cron
+- A firewall blocking all but the needed incoming ports
+    - ufw is an easy interface for the linux firewall
+- Protection against automatic attacks
+    - fail2ban scans logs and bans IPs which try to brute force their way into your server.
+    - Disable ssh login for root and switch from password to key based authentication.
+- Automated backups
+    - Most VPS hosting companies offer whole server backups for a small fee.
+    - Or run your own backup with something like borg.
+- A way to get notified if your disk fills up.
+    - If you send too much cat videos to your friends, Conduit might at some point become unable to
+      store any important messages.
@@ -0,0 +1,40 @@
+# About Matrix Homeservers
+
+Matrix homeservers manage its users chats. Every Matrix username includes the homserver it belongs to:
+`@alice:matrix.org` means that the `matrix.org` homeserver hosts a user called `@alice`.
+Every time someone chats with Alice, the `matrix.org` homeserver stores these messages.
+When `@alice:matrix.org` talks with `@adelaide:matrix.org`, that's easy. Both users use the same server.
+
+But how can `@bob:vector.tld`, who uses the `vector.tld` homeserver, exchange messages with `@alice:matrix.org`?
+This is where it get's a bit more complicated.
+
+## Matrix Homeserver discovery
+
+The Matrix specification specifies multiple ways how servers can discover and then talk to each other.
+Let's look at the most common one:
+
+### .well-known files
+
+At first, the only information a server has about a user (e.g. `@bob:vector.tld`) is its homeserver name: `vector.tld`.
+It then makes a HTTP GET request to `https://vector.tld/.well-known/matrix/server`.
+In the ideal case, this file contains a content like this: 
+
+```json
+{
+  "m.server": "matrix.vector.tld:443"
+}
+```
+
+This translates to: The matrix homeserver software for users with a username ending on `vector.tld`
+can be found at the address `matrix.vector.tld` at port 443 (which is the common port for HTTPS).
+
+The homeserver on it's quest to find `@bob:vector.tld` now contacts `matrix.vector.tld:443` and is then
+able to exchange chat messages with it.
+
+
+### Why so complicated?
+
+Organizations often don't want to run their Matrix server on the same machine that hosts their website,
+but `@foo:matrix.evil.corp` usernames are ugly and everyone wants to be `@foo:evil.corp`.
+
+To solve that problem, Matrix implements this extra step via a .well-known file or a DNS entry.
@@ -0,0 +1,20 @@
+site_name: Conduit Docs
+site_description: Conduit is a simple, fast and reliable chat server for the Matrix protocol
+theme:
+  name: material
+repo_url: https://gitlab.com/famedly/conduit
+nav:
+  - Home: index.md
+  - Installation:
+      - Prerequisites: installation/prerequisites.md
+      - Distribution Packages: installation/packages.md
+      - Manual: installation/manual.md
+      - Docker: installation/docker.md
+  - Configuration:
+      - Conduit.toml: configuration/conduit.toml.md
+      - Appservices: configuration/appservices.md
+  - Development:
+      - Basics: development/basics.md
+      - Cross compilation: development/cross-compilation.md
+      - Tests & CI: development/tests-ci.md
+
@@ -28,7 +28,6 @@ use ruma::{
    identifiers::RoomName,
    push, RoomAliasId, RoomId, RoomVersionId, UserId,
 };
-use serde_json::value::RawValue;
 use tracing::info;

 use register::RegistrationKind;
@@ -280,10 +279,7 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomCreate,
-                content: RawValue::from_string(
-                    serde_json::to_string(&content).expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
+                content: serde_json::to_value(content).expect("event is valid, we just created it"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -298,8 +294,7 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomMember,
-                content: RawValue::from_string(
-                    serde_json::to_string(&member::MemberEventContent {
+                content: serde_json::to_value(member::MemberEventContent {
                    membership: member::MembershipState::Join,
                    displayname: None,
                    avatar_url: None,
@@ -309,8 +304,6 @@ pub async fn register_route(
                    reason: None,
                })
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some(conduit_user.to_string()),
                redacts: None,
@@ -329,16 +322,13 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomPowerLevels,
-                content: RawValue::from_string(
-                    serde_json::to_string(
-                        &ruma::events::room::power_levels::PowerLevelsEventContent {
+                content: serde_json::to_value(
+                    ruma::events::room::power_levels::PowerLevelsEventContent {
                        users,
                        ..Default::default()
                    },
                )
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -353,13 +343,10 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomJoinRules,
-                content: RawValue::from_string(
-                    serde_json::to_string(&join_rules::JoinRulesEventContent::new(
+                content: serde_json::to_value(join_rules::JoinRulesEventContent::new(
                    join_rules::JoinRule::Invite,
                ))
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -374,13 +361,12 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomHistoryVisibility,
-                content: RawValue::from_string(
-                    serde_json::to_string(&history_visibility::HistoryVisibilityEventContent::new(
+                content: serde_json::to_value(
+                    history_visibility::HistoryVisibilityEventContent::new(
                        history_visibility::HistoryVisibility::Shared,
-                    ))
-                    .expect("event is valid, we just created it"),
+                    ),
                )
-                .expect("string is valid"),
+                .expect("event is valid, we just created it"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -395,13 +381,10 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomGuestAccess,
-                content: RawValue::from_string(
-                    serde_json::to_string(&guest_access::GuestAccessEventContent::new(
+                content: serde_json::to_value(guest_access::GuestAccessEventContent::new(
                    guest_access::GuestAccess::Forbidden,
                ))
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -419,11 +402,8 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomName,
-                content: RawValue::from_string(
-                    serde_json::to_string(&name::NameEventContent::new(Some(room_name)))
+                content: serde_json::to_value(name::NameEventContent::new(Some(room_name)))
                    .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -437,13 +417,10 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomTopic,
-                content: RawValue::from_string(
-                    serde_json::to_string(&topic::TopicEventContent {
+                content: serde_json::to_value(topic::TopicEventContent {
                    topic: format!("Manage {}", db.globals.server_name()),
                })
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -462,14 +439,11 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomCanonicalAlias,
-                content: RawValue::from_string(
-                    serde_json::to_string(&canonical_alias::CanonicalAliasEventContent {
+                content: serde_json::to_value(canonical_alias::CanonicalAliasEventContent {
                    alias: Some(alias.clone()),
                    alt_aliases: Vec::new(),
                })
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -486,8 +460,7 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomMember,
-                content: RawValue::from_string(
-                    serde_json::to_string(&member::MemberEventContent {
+                content: serde_json::to_value(member::MemberEventContent {
                    membership: member::MembershipState::Invite,
                    displayname: None,
                    avatar_url: None,
@@ -497,8 +470,6 @@ pub async fn register_route(
                    reason: None,
                })
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some(user_id.to_string()),
                redacts: None,
@@ -511,8 +482,7 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomMember,
-                content: RawValue::from_string(
-                    serde_json::to_string(&member::MemberEventContent {
+                content: serde_json::to_value(member::MemberEventContent {
                    membership: member::MembershipState::Join,
                    displayname: Some(displayname),
                    avatar_url: None,
@@ -522,8 +492,6 @@ pub async fn register_route(
                    reason: None,
                })
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some(user_id.to_string()),
                redacts: None,
@@ -538,11 +506,11 @@ pub async fn register_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomMessage,
-                content: RawValue::from_string(serde_json::to_string(&message::MessageEventContent::text_html(
+                content: serde_json::to_value(message::MessageEventContent::text_html(
                        "## Thank you for trying out Conduit!\n\nConduit is currently in Beta. This means you can join and participate in most Matrix rooms, but not all features are supported and you might run into bugs from time to time.\n\nHelpful links:\n> Website: https://conduit.rs\n> Git and Documentation: https://gitlab.com/famedly/conduit\n> Report issues: https://gitlab.com/famedly/conduit/-/issues\n\nHere are some rooms you can join (by typing the command):\n\nConduit room (Ask questions and get notified on updates):\n`/join #conduit:fachschaften.org`\n\nConduit lounge (Off-topic, only Conduit users are allowed to join)\n`/join #conduit-lounge:conduit.rs`".to_owned(),
                        "<h2>Thank you for trying out Conduit!</h2>\n<p>Conduit is currently in Beta. This means you can join and participate in most Matrix rooms, but not all features are supported and you might run into bugs from time to time.</p>\n<p>Helpful links:</p>\n<blockquote>\n<p>Website: https://conduit.rs<br>Git and Documentation: https://gitlab.com/famedly/conduit<br>Report issues: https://gitlab.com/famedly/conduit/-/issues</p>\n</blockquote>\n<p>Here are some rooms you can join (by typing the command):</p>\n<p>Conduit room (Ask questions and get notified on updates):<br><code>/join #conduit:fachschaften.org</code></p>\n<p>Conduit lounge (Off-topic, only Conduit users are allowed to join)<br><code>/join #conduit-lounge:conduit.rs</code></p>\n".to_owned(),
                ))
-                .expect("event is valid, we just created it")).expect("string is valid"),
+                .expect("event is valid, we just created it"),
                unsigned: None,
                state_key: None,
                redacts: None,
@@ -753,10 +721,7 @@ pub async fn deactivate_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomMember,
-                content: RawValue::from_string(
-                    serde_json::to_string(&event).expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
+                content: serde_json::to_value(event).expect("event is valid, we just created it"),
                unsigned: None,
                state_key: Some(sender_user.to_string()),
                redacts: None,
@@ -20,6 +20,7 @@ use ruma::{
        room::{avatar, canonical_alias, guest_access, history_visibility, name, topic},
        EventType,
    },
+    serde::Raw,
    ServerName, UInt,
 };
 use tracing::{info, warn};
@@ -216,8 +217,8 @@ pub(crate) async fn get_public_rooms_filtered_helper(
        }
    }

-    let mut all_rooms = db
-        .rooms
+    let mut all_rooms =
+        db.rooms
            .public_rooms()
            .map(|room_id| {
                let room_id = room_id?;
@@ -228,27 +229,29 @@ pub(crate) async fn get_public_rooms_filtered_helper(
                        .rooms
                        .room_state_get(&room_id, &EventType::RoomCanonicalAlias, "")?
                        .map_or(Ok::<_, Error>(None), |s| {
-                        Ok(
-                            serde_json::from_str::<canonical_alias::CanonicalAliasEventContent>(
-                                s.content.get(),
-                            )
+                            Ok(serde_json::from_value::<
+                                Raw<canonical_alias::CanonicalAliasEventContent>,
+                            >(s.content.clone())
+                            .expect("from_value::<Raw<..>> can never fail")
+                            .deserialize()
                            .map_err(|_| {
                                Error::bad_database("Invalid canonical alias event in database.")
                            })?
-                            .alias,
-                        )
+                            .alias)
                        })?,
                    name: db
                        .rooms
                        .room_state_get(&room_id, &EventType::RoomName, "")?
                        .map_or(Ok::<_, Error>(None), |s| {
-                        Ok(
-                            serde_json::from_str::<name::NameEventContent>(s.content.get())
+                            Ok(serde_json::from_value::<Raw<name::NameEventContent>>(
+                                s.content.clone(),
+                            )
+                            .expect("from_value::<Raw<..>> can never fail")
+                            .deserialize()
                            .map_err(|_| {
                                Error::bad_database("Invalid room name event in database.")
                            })?
-                                .name,
-                        )
+                            .name)
                        })?,
                    num_joined_members: db
                        .rooms
@@ -264,7 +267,11 @@ pub(crate) async fn get_public_rooms_filtered_helper(
                        .room_state_get(&room_id, &EventType::RoomTopic, "")?
                        .map_or(Ok::<_, Error>(None), |s| {
                            Ok(Some(
-                            serde_json::from_str::<topic::TopicEventContent>(s.content.get())
+                                serde_json::from_value::<Raw<topic::TopicEventContent>>(
+                                    s.content.clone(),
+                                )
+                                .expect("from_value::<Raw<..>> can never fail")
+                                .deserialize()
                                .map_err(|_| {
                                    Error::bad_database("Invalid room topic event in database.")
                                })?
@@ -275,9 +282,11 @@ pub(crate) async fn get_public_rooms_filtered_helper(
                        .rooms
                        .room_state_get(&room_id, &EventType::RoomHistoryVisibility, "")?
                        .map_or(Ok::<_, Error>(false), |s| {
-                        Ok(serde_json::from_str::<
-                                history_visibility::HistoryVisibilityEventContent,
-                            >(s.content.get())
+                            Ok(serde_json::from_value::<
+                                Raw<history_visibility::HistoryVisibilityEventContent>,
+                            >(s.content.clone())
+                            .expect("from_value::<Raw<..>> can never fail")
+                            .deserialize()
                            .map_err(|_| {
                                Error::bad_database(
                                    "Invalid room history visibility event in database.",
@@ -291,9 +300,11 @@ pub(crate) async fn get_public_rooms_filtered_helper(
                        .room_state_get(&room_id, &EventType::RoomGuestAccess, "")?
                        .map_or(Ok::<_, Error>(false), |s| {
                            Ok(
-                            serde_json::from_str::<guest_access::GuestAccessEventContent>(
-                                s.content.get(),
+                            serde_json::from_value::<Raw<guest_access::GuestAccessEventContent>>(
+                                s.content.clone(),
                            )
+                            .expect("from_value::<Raw<..>> can never fail")
+                            .deserialize()
                            .map_err(|_| {
                                Error::bad_database("Invalid room guest access event in database.")
                            })?
@@ -306,7 +317,11 @@ pub(crate) async fn get_public_rooms_filtered_helper(
                        .room_state_get(&room_id, &EventType::RoomAvatar, "")?
                        .map(|s| {
                            Ok::<_, Error>(
-                            serde_json::from_str::<avatar::AvatarEventContent>(s.content.get())
+                                serde_json::from_value::<Raw<avatar::AvatarEventContent>>(
+                                    s.content.clone(),
+                                )
+                                .expect("from_value::<Raw<..>> can never fail")
+                                .deserialize()
                                .map_err(|_| {
                                    Error::bad_database("Invalid room avatar event in database.")
                                })?
@@ -5,7 +5,6 @@ use crate::{
    server_server, utils, ConduitResult, Database, Error, Result, Ruma,
 };
 use member::{MemberEventContent, MembershipState};
-use rayon::prelude::*;
 use ruma::{
    api::{
        client::{
@@ -16,10 +15,7 @@ use ruma::{
                unban_user, IncomingThirdPartySigned,
            },
        },
-        federation::{
-            self,
-            membership::{create_invite, create_join_event},
-        },
+        federation::{self, membership::create_invite},
    },
    events::{
        pdu::Pdu,
@@ -30,13 +26,13 @@ use ruma::{
    state_res::{self, RoomVersion},
    uint, EventId, RoomId, RoomVersionId, ServerName, UserId,
 };
-use serde_json::value::RawValue;
 use std::{
-    collections::{BTreeMap, HashMap, HashSet},
+    collections::{hash_map::Entry, BTreeMap, HashMap, HashSet},
    convert::{TryFrom, TryInto},
    sync::{Arc, RwLock},
+    time::{Duration, Instant},
 };
-use tracing::{error, warn};
+use tracing::{debug, error, warn};

 #[cfg(feature = "conduit_bin")]
 use rocket::{get, post};
@@ -208,7 +204,7 @@ pub async fn kick_user_route(
 ) -> ConduitResult<kick_user::Response> {
    let sender_user = body.sender_user.as_ref().expect("user is authenticated");

-    let mut event = serde_json::from_str::<Raw<ruma::events::room::member::MemberEventContent>>(
+    let mut event = serde_json::from_value::<Raw<ruma::events::room::member::MemberEventContent>>(
        db.rooms
            .room_state_get(
                &body.room_id,
@@ -220,7 +216,7 @@ pub async fn kick_user_route(
                "Cannot kick member that's not in the room.",
            ))?
            .content
-            .get(),
+            .clone(),
    )
    .expect("Raw::from_value always works")
    .deserialize()
@@ -242,10 +238,7 @@ pub async fn kick_user_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomMember,
-            content: RawValue::from_string(
-                serde_json::to_string(&event).expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
+            content: serde_json::to_value(event).expect("event is valid, we just created it"),
            unsigned: None,
            state_key: Some(body.user_id.to_string()),
            redacts: None,
@@ -297,8 +290,11 @@ pub async fn ban_user_route(
                reason: None,
            }),
            |event| {
-                let mut event =
-                    serde_json::from_str::<member::MemberEventContent>(event.content.get())
+                let mut event = serde_json::from_value::<Raw<member::MemberEventContent>>(
+                    event.content.clone(),
+                )
+                .expect("Raw::from_value always works")
+                .deserialize()
                .map_err(|_| Error::bad_database("Invalid member event in database."))?;
                event.membership = ruma::events::room::member::MembershipState::Ban;
                Ok(event)
@@ -318,10 +314,7 @@ pub async fn ban_user_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomMember,
-            content: RawValue::from_string(
-                serde_json::to_string(&event).expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
+            content: serde_json::to_value(event).expect("event is valid, we just created it"),
            unsigned: None,
            state_key: Some(body.user_id.to_string()),
            redacts: None,
@@ -353,7 +346,7 @@ pub async fn unban_user_route(
 ) -> ConduitResult<unban_user::Response> {
    let sender_user = body.sender_user.as_ref().expect("user is authenticated");

-    let mut event = serde_json::from_str::<ruma::events::room::member::MemberEventContent>(
+    let mut event = serde_json::from_value::<Raw<ruma::events::room::member::MemberEventContent>>(
        db.rooms
            .room_state_get(
                &body.room_id,
@@ -365,8 +358,10 @@ pub async fn unban_user_route(
                "Cannot unban a user who is not banned.",
            ))?
            .content
-            .get(),
+            .clone(),
    )
+    .expect("from_value::<Raw<..>> can never fail")
+    .deserialize()
    .map_err(|_| Error::bad_database("Invalid member event in database."))?;

    event.membership = ruma::events::room::member::MembershipState::Leave;
@@ -384,10 +379,7 @@ pub async fn unban_user_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomMember,
-            content: RawValue::from_string(
-                serde_json::to_string(&event).expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
+            content: serde_json::to_value(event).expect("event is valid, we just created it"),
            unsigned: None,
            state_key: Some(body.user_id.to_string()),
            redacts: None,
@@ -571,7 +563,6 @@ async fn join_room_by_id_helper(
                    },
                )
                .await;
-            warn!("Make join done");

            make_join_response_and_server = make_join_response.map(|r| (r, remote_server));

@@ -667,193 +658,93 @@ async fn join_room_by_id_helper(
            )
            .await?;

-        warn!("Send join done");
-
        db.rooms.get_or_create_shortroomid(room_id, &db.globals)?;

        let pdu = PduEvent::from_id_val(&event_id, join_event.clone())
            .map_err(|_| Error::BadServerResponse("Invalid join event PDU."))?;

-        let pub_key_map = Arc::new(RwLock::new(BTreeMap::new()));
-        let missing_servers = Arc::new(RwLock::new(BTreeMap::new()));
+        let mut state = HashMap::new();
+        let pub_key_map = RwLock::new(BTreeMap::new());

-        let create_join_event::RoomState {
-            state: mut room_state_state,
-            auth_chain: mut room_state_auth_chain,
-        } = send_join_response.room_state;
+        server_server::fetch_join_signing_keys(
+            &send_join_response,
+            &room_version,
+            &pub_key_map,
+            db,
+        )
+        .await?;
+
+        for result in send_join_response
+            .room_state
+            .state
+            .iter()
+            .map(|pdu| validate_and_add_event_id(pdu, &room_version, &pub_key_map, db))
+        {
+            let (event_id, value) = match result {
+                Ok(t) => t,
+                Err(_) => continue,
+            };
+
+            let pdu = PduEvent::from_id_val(&event_id, value.clone()).map_err(|e| {
+                warn!("{:?}: {}", value, e);
+                Error::BadServerResponse("Invalid PDU in send_join response.")
+            })?;
+
+            db.rooms.add_pdu_outlier(&event_id, &value)?;
+            if let Some(state_key) = &pdu.state_key {
+                let shortstatekey =
+                    db.rooms
+                        .get_or_create_shortstatekey(&pdu.kind, state_key, &db.globals)?;
+                state.insert(shortstatekey, pdu.event_id.clone());
+            }
+        }
+
+        let incoming_shortstatekey = db.rooms.get_or_create_shortstatekey(
+            &pdu.kind,
+            pdu.state_key
+                .as_ref()
+                .expect("Pdu is a membership state event"),
+            &db.globals,
+        )?;
+
+        state.insert(incoming_shortstatekey, pdu.event_id.clone());

        let create_shortstatekey = db
            .rooms
            .get_shortstatekey(&EventType::RoomCreate, "")?
            .expect("Room exists");

-        let mut saw_create_event = false;
-
-        warn!("Parsing send join response state");
-        const CHUNK_SIZE: usize = 500;
-        let mut parsed_state = room_state_state
-            .par_chunks_mut(CHUNK_SIZE)
-            .filter_map(|pdus| {
-                let mut r = HashMap::with_capacity(CHUNK_SIZE);
-                for pdu in pdus {
-                    let (id, value) = get_event_id(&pdu, &room_version).ok()?;
-                    r.insert(id, value);
-                }
-
-                let mut missing_servers = missing_servers.write().unwrap();
-                let mut pub_key_map = pub_key_map.write().unwrap();
-                for (_, value) in &r {
-                    server_server::get_server_keys_from_cache(
-                        &value,
-                        &mut missing_servers,
-                        &mut pub_key_map,
-                        &db,
-                    )
-                    .ok()?;
-                }
-
-                Some(r)
-            })
-            .collect::<Vec<_>>();
-
-        warn!("Parsing send join response auth chain");
-        let mut parsed_chain = room_state_auth_chain
-            .par_chunks_mut(CHUNK_SIZE)
-            .filter_map(|pdus| {
-                let mut r = HashMap::with_capacity(CHUNK_SIZE);
-                for pdu in pdus {
-                    let (id, value) = get_event_id(&pdu, &room_version).ok()?;
-                    r.insert(id, value);
-                }
-
-                let mut missing_servers = missing_servers.write().unwrap();
-                let mut pub_key_map = pub_key_map.write().unwrap();
-                for (_, value) in &r {
-                    server_server::get_server_keys_from_cache(
-                        &value,
-                        &mut missing_servers,
-                        &mut pub_key_map,
-                        &db,
-                    )
-                    .ok()?;
-                }
-
-                Some(r)
-            })
-            .collect::<Vec<_>>();
-
-        warn!("Fetching send join signing keys");
-        server_server::fetch_join_signing_keys(missing_servers, &pub_key_map, db).await?;
-
-        warn!("Validating state");
-        parsed_state.par_iter_mut().for_each(|chunk| {
-            let mut bad_events = Vec::new();
-            for (event_id, value) in chunk.iter_mut() {
-                if let Err(e) = ruma::signatures::verify_event(
-                    &*pub_key_map.read().unwrap(),
-                    &value,
-                    &room_version,
-                ) {
-                    warn!("Event {} failed verification {:?} {}", event_id, value, e);
-                    bad_events.push(event_id.clone());
-                    continue;
-                }
-
-                value.insert(
-                    "event_id".to_owned(),
-                    CanonicalJsonValue::String(event_id.as_str().to_owned()),
-                );
-            }
-            for id in bad_events {
-                chunk.remove(&id);
-            }
-        });
-
-        warn!("Inserting state");
-        db.rooms
-            .add_pdu_outlier_batch(&mut parsed_state.iter().flatten())?;
-
-        warn!("Compressing state");
-        let state = parsed_state
-            .iter()
-            .flatten()
-            .map(|(event_id, value)| {
-                let kind = if let Some(s) = value.get("type").and_then(|s| s.as_str()) {
-                    s
-                } else {
-                    warn!("Event {} has no type: {:?}", event_id, value);
-                    return Ok(None);
-                };
-
-                if let Some(state_key) = value.get("state_key").and_then(|s| s.as_str()) {
-                    let shortstatekey = db.rooms.get_or_create_shortstatekey(
-                        &EventType::from(kind),
-                        state_key,
-                        &db.globals,
-                    )?;
-
-                    if shortstatekey == create_shortstatekey {
-                        saw_create_event = true;
-                    }
-
-                    Ok(Some(db.rooms.compress_state_event(
-                        shortstatekey,
-                        &event_id,
-                        &db.globals,
-                    )?))
-                } else {
-                    Ok(None)
-                }
-            })
-            .filter_map(|r| r.transpose())
-            .collect::<Result<HashSet<_>>>()?;
-
-        if !saw_create_event {
+        if state.get(&create_shortstatekey).is_none() {
            return Err(Error::BadServerResponse("State contained no create event."));
        }

-        warn!("Validating chain");
-        parsed_chain.par_iter_mut().for_each(|chunk| {
-            let mut bad_events = Vec::new();
-            for (event_id, value) in chunk.iter_mut() {
-                if let Err(e) = ruma::signatures::verify_event(
-                    &*pub_key_map.read().unwrap(),
-                    &value,
-                    &room_version,
-                ) {
-                    warn!("Event {} failed verification {:?} {}", event_id, value, e);
-                    bad_events.push(event_id.clone());
-                    continue;
-                }
-
-                value.insert(
-                    "event_id".to_owned(),
-                    CanonicalJsonValue::String(event_id.as_str().to_owned()),
-                );
-            }
-            for id in bad_events {
-                chunk.remove(&id);
-            }
-        });
-
-        warn!("Inserting chain");
-        db.rooms
-            .add_pdu_outlier_batch(&mut parsed_chain.iter().flatten())?;
-
-        warn!("Forcing state of room");
-        db.rooms.force_state_new(
+        db.rooms.force_state(
            room_id,
-            state,
-            &mut parsed_state.iter().flat_map(|m| m.values()),
+            state
+                .into_iter()
+                .map(|(k, id)| db.rooms.compress_state_event(k, &id, &db.globals))
+                .collect::<Result<HashSet<_>>>()?,
            db,
        )?;

+        for result in send_join_response
+            .room_state
+            .auth_chain
+            .iter()
+            .map(|pdu| validate_and_add_event_id(pdu, &room_version, &pub_key_map, db))
+        {
+            let (event_id, value) = match result {
+                Ok(t) => t,
+                Err(_) => continue,
+            };
+
+            db.rooms.add_pdu_outlier(&event_id, &value)?;
+        }
+
        // We append to state before appending the pdu, so we don't have a moment in time with the
        // pdu without it's state. This is okay because append_pdu can't fail.
-        warn!("Appending join event to state");
        let statehashid = db.rooms.append_to_state(&pdu, &db.globals)?;

-        warn!("Adding join event to db");
        db.rooms.append_pdu(
            &pdu,
            utils::to_canonical_object(&pdu).expect("Pdu is valid canonical object"),
@@ -861,7 +752,6 @@ async fn join_room_by_id_helper(
            db,
        )?;

-        warn!("Updating room state to join event");
        // We set the room state after inserting the pdu, so that we never have a moment in time
        // where events in the current room state do not exist
        db.rooms.set_room_state(room_id, statehashid)?;
@@ -879,10 +769,7 @@ async fn join_room_by_id_helper(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomMember,
-                content: RawValue::from_string(
-                    serde_json::to_string(&event).expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
+                content: serde_json::to_value(event).expect("event is valid, we just created it"),
                unsigned: None,
                state_key: Some(sender_user.to_string()),
                redacts: None,
@@ -901,15 +788,16 @@ async fn join_room_by_id_helper(
    Ok(join_room_by_id::Response::new(room_id.clone()).into())
 }

-fn get_event_id(
+fn validate_and_add_event_id(
    pdu: &Raw<Pdu>,
    room_version: &RoomVersionId,
+    pub_key_map: &RwLock<BTreeMap<String, BTreeMap<String, String>>>,
+    db: &Database,
 ) -> Result<(EventId, CanonicalJsonObject)> {
-    let value = serde_json::from_str::<CanonicalJsonObject>(pdu.json().get()).map_err(|e| {
-        warn!("Invalid PDU in server response: {:?}: {:?}", pdu, e);
+    let mut value = serde_json::from_str::<CanonicalJsonObject>(pdu.json().get()).map_err(|e| {
+        error!("Invalid PDU in server response: {:?}: {:?}", pdu, e);
        Error::BadServerResponse("Invalid PDU in server response")
    })?;
-
    let event_id = EventId::try_from(&*format!(
        "${}",
        ruma::signatures::reference_hash(&value, room_version)
@@ -917,6 +805,49 @@ fn get_event_id(
    ))
    .expect("ruma's reference hashes are valid event ids");

+    let back_off = |id| match db.globals.bad_event_ratelimiter.write().unwrap().entry(id) {
+        Entry::Vacant(e) => {
+            e.insert((Instant::now(), 1));
+        }
+        Entry::Occupied(mut e) => *e.get_mut() = (Instant::now(), e.get().1 + 1),
+    };
+
+    if let Some((time, tries)) = db
+        .globals
+        .bad_event_ratelimiter
+        .read()
+        .unwrap()
+        .get(&event_id)
+    {
+        // Exponential backoff
+        let mut min_elapsed_duration = Duration::from_secs(30) * (*tries) * (*tries);
+        if min_elapsed_duration > Duration::from_secs(60 * 60 * 24) {
+            min_elapsed_duration = Duration::from_secs(60 * 60 * 24);
+        }
+
+        if time.elapsed() < min_elapsed_duration {
+            debug!("Backing off from {}", event_id);
+            return Err(Error::BadServerResponse("bad event, still backing off"));
+        }
+    }
+
+    if let Err(e) = ruma::signatures::verify_event(
+        &*pub_key_map
+            .read()
+            .map_err(|_| Error::bad_database("RwLock is poisoned."))?,
+        &value,
+        room_version,
+    ) {
+        warn!("Event {} failed verification {:?} {}", event_id, pdu, e);
+        back_off(event_id);
+        return Err(Error::BadServerResponse("Event failed verification."));
+    }
+
+    value.insert(
+        "event_id".to_owned(),
+        CanonicalJsonValue::String(event_id.as_str().to_owned()),
+    );
+
    Ok((event_id, value))
 }

@@ -953,7 +884,7 @@ pub(crate) async fn invite_helper<'a>(
            let create_event_content = create_event
                .as_ref()
                .map(|create_event| {
-                    serde_json::from_str::<Raw<CreateEventContent>>(create_event.content.get())
+                    serde_json::from_value::<Raw<CreateEventContent>>(create_event.content.clone())
                        .expect("Raw::from_value always works.")
                        .deserialize()
                        .map_err(|e| {
@@ -979,8 +910,7 @@ pub(crate) async fn invite_helper<'a>(
            let room_version =
                RoomVersion::new(&room_version_id).expect("room version is supported");

-            let content = RawValue::from_string(
-                serde_json::to_string(&MemberEventContent {
+            let content = serde_json::to_value(MemberEventContent {
                avatar_url: None,
                displayname: None,
                is_direct: Some(is_direct),
@@ -989,9 +919,7 @@ pub(crate) async fn invite_helper<'a>(
                blurhash: None,
                reason: None,
            })
-                .expect("member event is valid value"),
-            )
-            .expect("string is valid");
+            .expect("member event is valid value");

            let state_key = user_id.to_string();
            let kind = EventType::RoomMember;
@@ -1018,7 +946,7 @@ pub(crate) async fn invite_helper<'a>(
                unsigned.insert("prev_content".to_owned(), prev_pdu.content.clone());
                unsigned.insert(
                    "prev_sender".to_owned(),
-                    serde_json::from_str(prev_pdu.sender.as_str()).expect("UserId is valid string"),
+                    serde_json::to_value(&prev_pdu.sender).expect("UserId::to_value always works"),
                );
            }

@@ -1031,7 +959,6 @@ pub(crate) async fn invite_helper<'a>(
                    .expect("time is valid"),
                kind,
                content,
-                parsed_content: RwLock::new(None),
                state_key: Some(state_key),
                prev_events,
                depth,
@@ -1040,20 +967,11 @@ pub(crate) async fn invite_helper<'a>(
                    .map(|(_, pdu)| pdu.event_id.clone())
                    .collect(),
                redacts: None,
-                unsigned: if unsigned.is_empty() {
-                    None
-                } else {
-                    Some(
-                        RawValue::from_string(
-                            serde_json::to_string(&unsigned).expect("to_string always works"),
-                        )
-                        .expect("string is valid"),
-                    )
-                },
+                unsigned,
                hashes: ruma::events::pdu::EventHash {
                    sha256: "aaa".to_owned(),
                },
-                signatures: None,
+                signatures: BTreeMap::new(),
            };

            let auth_check = state_res::auth_check(
@@ -1174,13 +1092,14 @@ pub(crate) async fn invite_helper<'a>(
            "Could not accept incoming PDU as timeline event.",
        ))?;

-        let servers = db
+        for server in db
            .rooms
            .room_servers(room_id)
            .filter_map(|r| r.ok())
-            .filter(|server| &**server != db.globals.server_name());
-
-        db.sending.send_pdu(servers, &pdu_id)?;
+            .filter(|server| &**server != db.globals.server_name())
+        {
+            db.sending.send_pdu(&server, &pdu_id)?;
+        }

        return Ok(());
    }
@@ -1198,8 +1117,7 @@ pub(crate) async fn invite_helper<'a>(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomMember,
-            content: RawValue::from_string(
-                serde_json::to_string(&member::MemberEventContent {
+            content: serde_json::to_value(member::MemberEventContent {
                membership: member::MembershipState::Invite,
                displayname: db.users.displayname(user_id)?,
                avatar_url: db.users.avatar_url(user_id)?,
@@ -1209,8 +1127,6 @@ pub(crate) async fn invite_helper<'a>(
                reason: None,
            })
            .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some(user_id.to_string()),
            redacts: None,
@@ -10,8 +10,8 @@ use ruma::{
        federation::{self, query::get_profile_information::v1::ProfileField},
    },
    events::EventType,
+    serde::Raw,
 };
-use serde_json::value::RawValue;
 use std::{convert::TryInto, sync::Arc};

 #[cfg(feature = "conduit_bin")]
@@ -45,10 +45,9 @@ pub async fn set_displayname_route(
            Ok::<_, Error>((
                PduBuilder {
                    event_type: EventType::RoomMember,
-                    content: RawValue::from_string(
-                        serde_json::to_string(&ruma::events::room::member::MemberEventContent {
+                    content: serde_json::to_value(ruma::events::room::member::MemberEventContent {
                        displayname: body.displayname.clone(),
-                            ..serde_json::from_str(
+                        ..serde_json::from_value::<Raw<_>>(
                            db.rooms
                                .room_state_get(
                                    &room_id,
@@ -62,13 +61,13 @@ pub async fn set_displayname_route(
                                    )
                                })?
                                .content
-                                    .get(),
+                                .clone(),
                        )
+                        .expect("from_value::<Raw<..>> can never fail")
+                        .deserialize()
                        .map_err(|_| Error::bad_database("Database contains invalid PDU."))?
                    })
                    .expect("event is valid, we just created it"),
-                    )
-                    .expect("string is valid"),
                    unsigned: None,
                    state_key: Some(sender_user.to_string()),
                    redacts: None,
@@ -191,10 +190,9 @@ pub async fn set_avatar_url_route(
            Ok::<_, Error>((
                PduBuilder {
                    event_type: EventType::RoomMember,
-                    content: RawValue::from_string(
-                        serde_json::to_string(&ruma::events::room::member::MemberEventContent {
+                    content: serde_json::to_value(ruma::events::room::member::MemberEventContent {
                        avatar_url: body.avatar_url.clone(),
-                            ..serde_json::from_str(
+                        ..serde_json::from_value::<Raw<_>>(
                            db.rooms
                                .room_state_get(
                                    &room_id,
@@ -208,13 +206,13 @@ pub async fn set_avatar_url_route(
                                    )
                                })?
                                .content
-                                    .get(),
+                                .clone(),
                        )
+                        .expect("from_value::<Raw<..>> can never fail")
+                        .deserialize()
                        .map_err(|_| Error::bad_database("Database contains invalid PDU."))?
                    })
                    .expect("event is valid, we just created it"),
-                    )
-                    .expect("string is valid"),
                    unsigned: None,
                    state_key: Some(sender_user.to_string()),
                    redacts: None,
@@ -8,7 +8,6 @@ use ruma::{

 #[cfg(feature = "conduit_bin")]
 use rocket::put;
-use serde_json::value::RawValue;

 /// # `PUT /_matrix/client/r0/rooms/{roomId}/redact/{eventId}/{txnId}`
 ///
@@ -39,13 +38,10 @@ pub async fn redact_event_route(
    let event_id = db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomRedaction,
-            content: RawValue::from_string(
-                serde_json::to_string(&redaction::RedactionEventContent {
+            content: serde_json::to_value(redaction::RedactionEventContent {
                reason: body.reason.clone(),
            })
            .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: None,
            redacts: Some(body.event_id.clone()),
@@ -11,9 +11,9 @@ use ruma::{
        room::{guest_access, history_visibility, join_rules, member, name, topic},
        EventType,
    },
+    serde::Raw,
    RoomAliasId, RoomId, RoomVersionId,
 };
-use serde_json::value::RawValue;
 use std::{cmp::max, collections::BTreeMap, convert::TryFrom, sync::Arc};
 use tracing::{info, warn};

@@ -101,10 +101,7 @@ pub async fn create_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomCreate,
-            content: RawValue::from_string(
-                serde_json::to_string(&content).expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
+            content: serde_json::to_value(content).expect("event is valid, we just created it"),
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -119,8 +116,7 @@ pub async fn create_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomMember,
-            content: RawValue::from_string(
-                serde_json::to_string(&member::MemberEventContent {
+            content: serde_json::to_value(member::MemberEventContent {
                membership: member::MembershipState::Join,
                displayname: db.users.displayname(sender_user)?,
                avatar_url: db.users.avatar_url(sender_user)?,
@@ -130,8 +126,6 @@ pub async fn create_room_route(
                reason: None,
            })
            .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some(sender_user.to_string()),
            redacts: None,
@@ -186,8 +180,7 @@ pub async fn create_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomPowerLevels,
-            content: RawValue::from_string(power_levels_content.to_string())
-                .expect("string is valid"),
+            content: power_levels_content,
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -203,16 +196,13 @@ pub async fn create_room_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomCanonicalAlias,
-                content: RawValue::from_string(
-                    serde_json::to_string(
-                        &ruma::events::room::canonical_alias::CanonicalAliasEventContent {
+                content: serde_json::to_value(
+                    ruma::events::room::canonical_alias::CanonicalAliasEventContent {
                        alias: Some(room_alias_id.clone()),
                        alt_aliases: vec![],
                    },
                )
                .expect("We checked that alias earlier, it must be fine"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -230,15 +220,17 @@ pub async fn create_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomJoinRules,
-            content: RawValue::from_string(
-                serde_json::to_string(&join_rules::JoinRulesEventContent::new(match preset {
-                    create_room::RoomPreset::PublicChat => join_rules::JoinRule::Public,
-                    // according to spec "invite" is the default
-                    _ => join_rules::JoinRule::Invite,
-                }))
-                .expect("event is valid, we just created it"),
+            content: match preset {
+                create_room::RoomPreset::PublicChat => serde_json::to_value(
+                    join_rules::JoinRulesEventContent::new(join_rules::JoinRule::Public),
                )
-            .expect("string is valid"),
+                .expect("event is valid, we just created it"),
+                // according to spec "invite" is the default
+                _ => serde_json::to_value(join_rules::JoinRulesEventContent::new(
+                    join_rules::JoinRule::Invite,
+                ))
+                .expect("event is valid, we just created it"),
+            },
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -253,13 +245,10 @@ pub async fn create_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomHistoryVisibility,
-            content: RawValue::from_string(
-                serde_json::to_string(&history_visibility::HistoryVisibilityEventContent::new(
+            content: serde_json::to_value(history_visibility::HistoryVisibilityEventContent::new(
                history_visibility::HistoryVisibility::Shared,
            ))
            .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -274,14 +263,18 @@ pub async fn create_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomGuestAccess,
-            content: RawValue::from_string(
-                serde_json::to_string(&guest_access::GuestAccessEventContent::new(match preset {
-                    create_room::RoomPreset::PublicChat => guest_access::GuestAccess::Forbidden,
-                    _ => guest_access::GuestAccess::CanJoin,
-                }))
+            content: match preset {
+                create_room::RoomPreset::PublicChat => {
+                    serde_json::to_value(guest_access::GuestAccessEventContent::new(
+                        guest_access::GuestAccess::Forbidden,
+                    ))
+                    .expect("event is valid, we just created it")
+                }
+                _ => serde_json::to_value(guest_access::GuestAccessEventContent::new(
+                    guest_access::GuestAccess::CanJoin,
+                ))
                .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
+            },
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -313,11 +306,8 @@ pub async fn create_room_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomName,
-                content: RawValue::from_string(
-                    serde_json::to_string(&name::NameEventContent::new(Some(name.clone())))
+                content: serde_json::to_value(name::NameEventContent::new(Some(name.clone())))
                    .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -333,13 +323,10 @@ pub async fn create_room_route(
        db.rooms.build_and_append_pdu(
            PduBuilder {
                event_type: EventType::RoomTopic,
-                content: RawValue::from_string(
-                    serde_json::to_string(&topic::TopicEventContent {
+                content: serde_json::to_value(topic::TopicEventContent {
                    topic: topic.clone(),
                })
                .expect("event is valid, we just created it"),
-                )
-                .expect("string is valid"),
                unsigned: None,
                state_key: Some("".to_owned()),
                redacts: None,
@@ -490,14 +477,11 @@ pub async fn upgrade_room_route(
    let tombstone_event_id = db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomTombstone,
-            content: RawValue::from_string(
-                serde_json::to_string(&ruma::events::room::tombstone::TombstoneEventContent {
+            content: serde_json::to_value(ruma::events::room::tombstone::TombstoneEventContent {
                body: "This room has been replaced".to_string(),
                replacement_room: replacement_room.clone(),
            })
            .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -521,13 +505,15 @@ pub async fn upgrade_room_route(
    let state_lock = mutex_state.lock().await;

    // Get the old room federations status
-    let federate = serde_json::from_str::<ruma::events::room::create::CreateEventContent>(
+    let federate = serde_json::from_value::<Raw<ruma::events::room::create::CreateEventContent>>(
        db.rooms
            .room_state_get(&body.room_id, &EventType::RoomCreate, "")?
            .ok_or_else(|| Error::bad_database("Found room without m.room.create event."))?
            .content
-            .get(),
+            .clone(),
    )
+    .expect("Raw::from_value always works")
+    .deserialize()
    .map_err(|_| Error::bad_database("Invalid room event in database."))?
    .federate;

@@ -547,11 +533,8 @@ pub async fn upgrade_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomCreate,
-            content: RawValue::from_string(
-                serde_json::to_string(&create_event_content)
+            content: serde_json::to_value(create_event_content)
                .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -566,8 +549,7 @@ pub async fn upgrade_room_route(
    db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomMember,
-            content: RawValue::from_string(
-                serde_json::to_string(&member::MemberEventContent {
+            content: serde_json::to_value(member::MemberEventContent {
                membership: member::MembershipState::Join,
                displayname: db.users.displayname(sender_user)?,
                avatar_url: db.users.avatar_url(sender_user)?,
@@ -577,8 +559,6 @@ pub async fn upgrade_room_route(
                reason: None,
            })
            .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some(sender_user.to_string()),
            redacts: None,
@@ -605,7 +585,7 @@ pub async fn upgrade_room_route(
    // Replicate transferable state events to the new room
    for event_type in transferable_state_events {
        let event_content = match db.rooms.room_state_get(&body.room_id, &event_type, "")? {
-            Some(v) => v.content,
+            Some(v) => v.content.clone(),
            None => continue, // Skipping missing events.
        };

@@ -632,13 +612,15 @@ pub async fn upgrade_room_route(

    // Get the old room power levels
    let mut power_levels_event_content =
-        serde_json::from_str::<ruma::events::room::power_levels::PowerLevelsEventContent>(
+        serde_json::from_value::<Raw<ruma::events::room::power_levels::PowerLevelsEventContent>>(
            db.rooms
                .room_state_get(&body.room_id, &EventType::RoomPowerLevels, "")?
                .ok_or_else(|| Error::bad_database("Found room without m.room.create event."))?
                .content
-                .get(),
+                .clone(),
        )
+        .expect("database contains invalid PDU")
+        .deserialize()
        .map_err(|_| Error::bad_database("Invalid room event in database."))?;

    // Setting events_default and invite to the greater of 50 and users_default + 1
@@ -653,11 +635,8 @@ pub async fn upgrade_room_route(
    let _ = db.rooms.build_and_append_pdu(
        PduBuilder {
            event_type: EventType::RoomPowerLevels,
-            content: RawValue::from_string(
-                serde_json::to_string(&power_levels_event_content)
+            content: serde_json::to_value(power_levels_event_content)
                .expect("event is valid, we just created it"),
-            )
-            .expect("string is valid"),
            unsigned: None,
            state_key: Some("".to_owned()),
            redacts: None,
@@ -112,7 +112,7 @@ pub async fn get_state_events_route(
            db.rooms
                .room_state_get(&body.room_id, &EventType::RoomHistoryVisibility, "")?
                .map(|event| {
-                    serde_json::from_str::<HistoryVisibilityEventContent>(event.content.get())
+                    serde_json::from_value::<HistoryVisibilityEventContent>(event.content.clone())
                        .map_err(|_| {
                            Error::bad_database(
                                "Invalid room history visibility event in database.",
@@ -164,7 +164,7 @@ pub async fn get_state_events_for_key_route(
            db.rooms
                .room_state_get(&body.room_id, &EventType::RoomHistoryVisibility, "")?
                .map(|event| {
-                    serde_json::from_str::<HistoryVisibilityEventContent>(event.content.get())
+                    serde_json::from_value::<HistoryVisibilityEventContent>(event.content.clone())
                        .map_err(|_| {
                            Error::bad_database(
                                "Invalid room history visibility event in database.",
@@ -190,7 +190,7 @@ pub async fn get_state_events_for_key_route(
        ))?;

    Ok(get_state_events_for_key::Response {
-        content: serde_json::from_str(event.content.get())
+        content: serde_json::from_value(event.content.clone())
            .map_err(|_| Error::bad_database("Invalid event content in database"))?,
    }
    .into())
@@ -220,7 +220,7 @@ pub async fn get_state_events_for_empty_key_route(
            db.rooms
                .room_state_get(&body.room_id, &EventType::RoomHistoryVisibility, "")?
                .map(|event| {
-                    serde_json::from_str::<HistoryVisibilityEventContent>(event.content.get())
+                    serde_json::from_value::<HistoryVisibilityEventContent>(event.content.clone())
                        .map_err(|_| {
                            Error::bad_database(
                                "Invalid room history visibility event in database.",
@@ -246,7 +246,7 @@ pub async fn get_state_events_for_empty_key_route(
        ))?;

    Ok(get_state_events_for_key::Response {
-        content: serde_json::from_str(event.content.get())
+        content: serde_json::from_value(event.content.clone())
            .map_err(|_| Error::bad_database("Invalid event content in database"))?,
    }
    .into())
@@ -287,9 +287,9 @@ async fn sync_helper(
                    .filter_map(|pdu| pdu.ok()) // Ignore all broken pdus
                    .filter(|(_, pdu)| pdu.kind == EventType::RoomMember)
                    .map(|(_, pdu)| {
-                        let content = serde_json::from_str::<
+                        let content = serde_json::from_value::<
                            ruma::events::room::member::MemberEventContent,
-                        >(pdu.content.get())
+                        >(pdu.content.clone())
                        .map_err(|_| Error::bad_database("Invalid member event in database."))?;

                        if let Some(state_key) = &pdu.state_key {
@@ -371,9 +371,11 @@ async fn sync_helper(
                    sender_user.as_str(),
                )?
                .and_then(|pdu| {
-                    serde_json::from_str::<ruma::events::room::member::MemberEventContent>(
-                        pdu.content.get(),
+                    serde_json::from_value::<Raw<ruma::events::room::member::MemberEventContent>>(
+                        pdu.content.clone(),
                    )
+                    .expect("Raw::from_value always works")
+                    .deserialize()
                    .map_err(|_| Error::bad_database("Invalid PDU in database."))
                    .ok()
                });
@@ -430,9 +432,11 @@ async fn sync_helper(
                            continue;
                        }

-                        let new_membership = serde_json::from_str::<
-                            ruma::events::room::member::MemberEventContent,
-                        >(state_event.content.get())
+                        let new_membership = serde_json::from_value::<
+                            Raw<ruma::events::room::member::MemberEventContent>,
+                        >(state_event.content.clone())
+                        .expect("Raw::from_value always works")
+                        .deserialize()
                        .map_err(|_| Error::bad_database("Invalid PDU in database."))?
                        .membership;

@@ -9,7 +9,6 @@ use ruma::{
    events::{room::message, EventType},
    UserId,
 };
-use serde_json::value::RawValue;
 use tokio::sync::{MutexGuard, RwLock, RwLockReadGuard};
 use tracing::warn;

@@ -67,11 +66,8 @@ impl Admin {
                    .build_and_append_pdu(
                        PduBuilder {
                            event_type: EventType::RoomMessage,
-                            content: RawValue::from_string(
-                                serde_json::to_string(&message)
+                            content: serde_json::to_value(message)
                                .expect("event is valid, we just created it"),
-                            )
-                            .expect("string is valid"),
                            unsigned: None,
                            state_key: None,
                            redacts: None,
@@ -9,10 +9,8 @@ use ruma::{
        },
        IncomingResponse, OutgoingRequest, SendAccessToken,
    },
-    events::{
-        room::{name::NameEventContent, power_levels::PowerLevelsEventContent},
-        AnySyncRoomEvent, EventType,
-    },
+    events::{room::power_levels::PowerLevelsEventContent, AnySyncRoomEvent, EventType},
+    identifiers::RoomName,
    push::{Action, PushConditionRoomCtx, PushFormat, Ruleset, Tweak},
    serde::Raw,
    uint, RoomId, UInt, UserId,
@@ -183,7 +181,7 @@ pub async fn send_push_notice(
        .rooms
        .room_state_get(&pdu.room_id, &EventType::RoomPowerLevels, "")?
        .map(|ev| {
-            serde_json::from_str(ev.content.get())
+            serde_json::from_value(ev.content.clone())
                .map_err(|_| Error::bad_database("invalid m.room.power_levels event"))
        })
        .transpose()?
@@ -320,18 +318,16 @@ async fn send_notice(

        let user_name = db.users.displayname(&event.sender)?;
        notifi.sender_display_name = user_name.as_deref();
-
-        let room_name = if let Some(room_name_pdu) =
-            db.rooms
+        let room_name = db
+            .rooms
            .room_state_get(&event.room_id, &EventType::RoomName, "")?
-        {
-            serde_json::from_str::<NameEventContent>(room_name_pdu.content.get())
-                .map_err(|_| Error::bad_database("Invalid room name event in database."))?
-                .name
-        } else {
-            None
-        };
-
+            .map(|pdu| match pdu.content.get("name") {
+                Some(serde_json::Value::String(s)) => {
+                    Some(Box::<RoomName>::try_from(&**s).expect("room name is valid"))
+                }
+                _ => None,
+            })
+            .flatten();
        notifi.room_name = room_name.as_deref();

        send_request(
@@ -2,7 +2,6 @@ mod edus;

 pub use edus::RoomEdus;
 use member::MembershipState;
-use serde_json::value::RawValue;

 use crate::{pdu::PduBuilder, server_server, utils, Database, Error, PduEvent, Result};
 use lru_cache::LruCache;
@@ -97,7 +96,7 @@ pub struct Rooms {
    /// RoomId + EventId -> Parent PDU EventId.
    pub(super) referencedevents: Arc<dyn Tree>,

-    pub(super) pdu_cache: Mutex<LruCache<EventId, Vec<u8>>>,
+    pub(super) pdu_cache: Mutex<LruCache<EventId, Arc<PduEvent>>>,
    pub(super) shorteventid_cache: Mutex<LruCache<u64, Arc<EventId>>>,
    pub(super) auth_chain_cache: Mutex<LruCache<Vec<u64>, Arc<HashSet<u64>>>>,
    pub(super) eventidshort_cache: Mutex<LruCache<EventId, u64>>,
@@ -138,7 +137,7 @@ impl Rooms {
    pub fn state_full(
        &self,
        shortstatehash: u64,
-    ) -> Result<HashMap<(EventType, String), PduEvent>> {
+    ) -> Result<HashMap<(EventType, String), Arc<PduEvent>>> {
        let full_state = self
            .load_shortstatehash_info(shortstatehash)?
            .pop()
@@ -200,7 +199,7 @@ impl Rooms {
        shortstatehash: u64,
        event_type: &EventType,
        state_key: &str,
-    ) -> Result<Option<PduEvent>> {
+    ) -> Result<Option<Arc<PduEvent>>> {
        self.state_get_id(shortstatehash, event_type, state_key)?
            .map_or(Ok(None), |event_id| self.get_pdu(&event_id))
    }
@@ -244,8 +243,8 @@ impl Rooms {
        kind: &EventType,
        sender: &UserId,
        state_key: Option<&str>,
-        content: &serde_json::value::RawValue,
-    ) -> Result<StateMap<PduEvent>> {
+        content: &serde_json::Value,
+    ) -> Result<StateMap<Arc<PduEvent>>> {
        let shortstatehash =
            if let Some(current_shortstatehash) = self.current_shortstatehash(room_id)? {
                current_shortstatehash
@@ -253,12 +252,7 @@ impl Rooms {
                return Ok(HashMap::new());
            };

-        let auth_events = state_res::auth_types_for_event(
-            kind,
-            sender,
-            state_key,
-            &serde_json::from_str(content.get()).expect("RawValue is valid json"),
-        );
+        let auth_events = state_res::auth_types_for_event(kind, sender, state_key, content);

        let mut sauthevents = auth_events
            .into_iter()
@@ -335,75 +329,6 @@ impl Rooms {
            .transpose()
    }

-    /// Force the creation of a new StateHash and insert it into the db. This version should be
-    /// used when creating new rooms.
-    ///
-    /// Whatever `state` is supplied to `force_state` becomes the new current room state snapshot.
-    #[tracing::instrument(skip(self, new_state_ids_compressed, state_values, db))]
-    pub fn force_state_new(
-        &self,
-        room_id: &RoomId,
-        new_state_ids_compressed: HashSet<CompressedStateEvent>,
-        state_values: &mut dyn Iterator<Item = &CanonicalJsonObject>,
-        db: &Database,
-    ) -> Result<()> {
-        let state_hash = self.calculate_hash(
-            &new_state_ids_compressed
-                .iter()
-                .map(|bytes| &bytes[..])
-                .collect::<Vec<_>>(),
-        );
-
-        let (new_shortstatehash, _) =
-            self.get_or_create_shortstatehash(&state_hash, &db.globals)?;
-
-        self.save_state_from_diff(
-            new_shortstatehash,
-            new_state_ids_compressed.clone(),
-            HashSet::new(),
-            2, // every state change is 2 event changes on average
-            Vec::new(),
-        )?;
-
-        for pdu in state_values {
-            if pdu.get("type").and_then(|val| val.as_str()) == Some("m.room.member") {
-                let content = pdu
-                    .get("content")
-                    .and_then(|o| o.as_object())
-                    .ok_or_else(|| Error::bad_database("Invalid content in pdu."))?;
-                if let Some(membership) = content.get("membership").and_then(|membership| {
-                    serde_json::from_str::<member::MembershipState>(
-                        &serde_json::to_string(membership).expect("json is valid string"),
-                    )
-                    .ok()
-                }) {
-                    if let Some(state_key) = pdu
-                        .get("state_key")
-                        .and_then(|o| o.as_str())
-                        .and_then(|state_key| UserId::try_from(state_key).ok())
-                    {
-                        if let Some(sender) = pdu
-                            .get("sender")
-                            .and_then(|o| o.as_str())
-                            .and_then(|state_key| UserId::try_from(state_key).ok())
-                        {
-                            self.update_membership(
-                                room_id, &state_key, membership, &sender, None, db, false,
-                            )?;
-                        }
-                    }
-                }
-            }
-        }
-
-        self.update_joined_count(room_id, db)?;
-
-        self.roomid_shortstatehash
-            .insert(room_id.as_bytes(), &new_shortstatehash.to_be_bytes())?;
-
-        Ok(())
-    }
-
    /// Force the creation of a new StateHash and insert it into the db.
    ///
    /// Whatever `state` is supplied to `force_state` becomes the new current room state snapshot.
@@ -468,28 +393,29 @@ impl Rooms {
        }) {
            if let Some(pdu) = self.get_pdu_json(&event_id)? {
                if pdu.get("type").and_then(|val| val.as_str()) == Some("m.room.member") {
-                    let content = pdu
-                        .get("content")
-                        .and_then(|o| o.as_object())
-                        .ok_or_else(|| Error::bad_database("Invalid content in pdu."))?;
-                    if let Some(membership) = content.get("membership").and_then(|membership| {
-                        serde_json::from_str::<member::MembershipState>(
-                            &serde_json::to_string(membership).expect("json is valid string"),
+                    if let Ok(pdu) = serde_json::from_value::<PduEvent>(
+                        serde_json::to_value(&pdu).expect("CanonicalJsonObj is a valid JsonValue"),
+                    ) {
+                        if let Some(membership) =
+                            pdu.content.get("membership").and_then(|membership| {
+                                serde_json::from_value::<member::MembershipState>(
+                                    membership.clone(),
                                )
                                .ok()
-                    }) {
-                        if let Some(state_key) = pdu
-                            .get("state_key")
-                            .and_then(|o| o.as_str())
-                            .and_then(|state_key| UserId::try_from(state_key).ok())
+                            })
                        {
-                            if let Some(sender) = pdu
-                                .get("sender")
-                                .and_then(|o| o.as_str())
+                            if let Some(state_key) = pdu
+                                .state_key
                                .and_then(|state_key| UserId::try_from(state_key).ok())
                            {
                                self.update_membership(
-                                    room_id, &state_key, membership, &sender, None, db, false,
+                                    room_id,
+                                    &state_key,
+                                    membership,
+                                    &pdu.sender,
+                                    None,
+                                    db,
+                                    false,
                                )?;
                            }
                        }
@@ -989,7 +915,7 @@ impl Rooms {
    pub fn room_state_full(
        &self,
        room_id: &RoomId,
-    ) -> Result<HashMap<(EventType, String), PduEvent>> {
+    ) -> Result<HashMap<(EventType, String), Arc<PduEvent>>> {
        if let Some(current_shortstatehash) = self.current_shortstatehash(room_id)? {
            self.state_full(current_shortstatehash)
        } else {
@@ -1019,7 +945,7 @@ impl Rooms {
        room_id: &RoomId,
        event_type: &EventType,
        state_key: &str,
-    ) -> Result<Option<PduEvent>> {
+    ) -> Result<Option<Arc<PduEvent>>> {
        if let Some(current_shortstatehash) = self.current_shortstatehash(room_id)? {
            self.state_get(current_shortstatehash, event_type, state_key)
        } else {
@@ -1147,11 +1073,9 @@ impl Rooms {
    ///
    /// Checks the `eventid_outlierpdu` Tree if not found in the timeline.
    #[tracing::instrument(skip(self))]
-    pub fn get_pdu(&self, event_id: &EventId) -> Result<Option<PduEvent>> {
+    pub fn get_pdu(&self, event_id: &EventId) -> Result<Option<Arc<PduEvent>>> {
        if let Some(p) = self.pdu_cache.lock().unwrap().get_mut(event_id) {
-            return Ok(Some(
-                serde_json::from_slice(p).expect("pdus in cache are valid"),
-            ));
+            return Ok(Some(Arc::clone(p)));
        }

        if let Some(pdu) = self
@@ -1168,13 +1092,18 @@ impl Rooms {
                    })?))
                },
            )?
+            .map(|pdu| {
+                serde_json::from_slice(&pdu)
+                    .map_err(|_| Error::bad_database("Invalid PDU in db."))
+                    .map(Arc::new)
+            })
+            .transpose()?
        {
-            let parsed =
-                serde_json::from_slice(&pdu).map_err(|_| Error::bad_database("Invalid PDU in db."));
-
-            self.pdu_cache.lock().unwrap().insert(event_id.clone(), pdu);
-
-            parsed
+            self.pdu_cache
+                .lock()
+                .unwrap()
+                .insert(event_id.clone(), Arc::clone(&pdu));
+            Ok(Some(pdu))
        } else {
            Ok(None)
        }
@@ -1299,23 +1228,6 @@ impl Rooms {
        )
    }

-    /// Append the PDU as an outlier.
-    ///
-    /// Any event given to this will be processed (state-res) on another thread.
-    #[tracing::instrument(skip(self, batch))]
-    pub fn add_pdu_outlier_batch(
-        &self,
-        batch: &mut dyn Iterator<Item = (&EventId, &CanonicalJsonObject)>,
-    ) -> Result<()> {
-        self.eventid_outlierpdu
-            .insert_batch(&mut batch.map(|(event_id, pdu)| {
-                (
-                    event_id.as_bytes().to_vec(),
-                    serde_json::to_vec(&pdu).expect("CanonicalJsonObject is valid"),
-                )
-            }))
-    }
-
    #[tracing::instrument(skip(self))]
    pub fn mark_event_soft_failed(&self, event_id: &EventId) -> Result<()> {
        self.softfailedeventids.insert(event_id.as_bytes(), &[])
@@ -1417,7 +1329,7 @@ impl Rooms {
            .rooms
            .room_state_get(&pdu.room_id, &EventType::RoomPowerLevels, "")?
            .map(|ev| {
-                serde_json::from_str(ev.content.get())
+                serde_json::from_value(ev.content.clone())
                    .map_err(|_| Error::bad_database("invalid m.room.power_levels event"))
            })
            .transpose()?
@@ -1496,11 +1408,8 @@ impl Rooms {
                    let target_user_id = UserId::try_from(state_key.clone())
                        .expect("This state_key was previously validated");

-                    let content = serde_json::from_str::<serde_json::Value>(pdu.content.get())
-                        .map_err(|_| Error::bad_database("Invalid content in pdu."))?;
-
                    let membership = serde_json::from_value::<member::MembershipState>(
-                        content
+                        pdu.content
                            .get("membership")
                            .ok_or(Error::BadRequest(
                                ErrorKind::InvalidParam,
@@ -1538,10 +1447,7 @@ impl Rooms {
                }
            }
            EventType::RoomMessage => {
-                let content = serde_json::from_str::<serde_json::Value>(pdu.content.get())
-                    .map_err(|_| Error::bad_database("Invalid content in pdu."))?;
-
-                if let Some(body) = content.get("body").and_then(|b| b.as_str()) {
+                if let Some(body) = pdu.content.get("body").and_then(|b| b.as_str()) {
                    let mut batch = body
                        .split_terminator(|c: char| !c.is_alphanumeric())
                        .filter(|s| !s.is_empty())
@@ -2055,7 +1961,7 @@ impl Rooms {
        let create_event_content = create_event
            .as_ref()
            .map(|create_event| {
-                serde_json::from_str::<Raw<CreateEventContent>>(create_event.content.get())
+                serde_json::from_value::<Raw<CreateEventContent>>(create_event.content.clone())
                    .expect("Raw::from_value always works.")
                    .deserialize()
                    .map_err(|e| {
@@ -2094,10 +2000,7 @@ impl Rooms {
        let mut unsigned = unsigned.unwrap_or_default();
        if let Some(state_key) = &state_key {
            if let Some(prev_pdu) = self.room_state_get(room_id, &event_type, state_key)? {
-                unsigned.insert(
-                    "prev_content".to_owned(),
-                    serde_json::from_str(prev_pdu.content.get()).expect("string is valid json"),
-                );
+                unsigned.insert("prev_content".to_owned(), prev_pdu.content.clone());
                unsigned.insert(
                    "prev_sender".to_owned(),
                    serde_json::to_value(&prev_pdu.sender).expect("UserId::to_value always works"),
@@ -2114,7 +2017,6 @@ impl Rooms {
                .expect("time is valid"),
            kind: event_type,
            content,
-            parsed_content: RwLock::new(None),
            state_key,
            prev_events,
            depth,
@@ -2123,20 +2025,11 @@ impl Rooms {
                .map(|(_, pdu)| pdu.event_id.clone())
                .collect(),
            redacts,
-            unsigned: if unsigned.is_empty() {
-                None
-            } else {
-                Some(
-                    RawValue::from_string(
-                        serde_json::to_string(&unsigned).expect("to_string always works"),
-                    )
-                    .expect("string is valid"),
-                )
-            },
+            unsigned,
            hashes: ruma::events::pdu::EventHash {
                sha256: "aaa".to_owned(),
            },
-            signatures: None,
+            signatures: BTreeMap::new(),
        };

        let auth_check = state_res::auth_check(
@@ -2211,12 +2104,13 @@ impl Rooms {
        // where events in the current room state do not exist
        self.set_room_state(room_id, statehashid)?;

-        let servers = self
+        for server in self
            .room_servers(room_id)
            .filter_map(|r| r.ok())
-            .filter(|server| &**server != db.globals.server_name());
-
-        db.sending.send_pdu(servers, &pdu_id)?;
+            .filter(|server| &**server != db.globals.server_name())
+        {
+            db.sending.send_pdu(&server, &pdu_id)?;
+        }

        for appservice in db.appservice.all()? {
            if self.appservice_in_room(room_id, &appservice, db)? {
@@ -2312,7 +2206,7 @@ impl Rooms {
                let mut pdu = serde_json::from_slice::<PduEvent>(&v)
                    .map_err(|_| Error::bad_database("PDU in db is invalid."))?;
                if pdu.sender != user_id {
-                    pdu.remove_transaction_id()?;
+                    pdu.unsigned.remove("transaction_id");
                }
                Ok((pdu_id, pdu))
            }))
@@ -2349,7 +2243,7 @@ impl Rooms {
                let mut pdu = serde_json::from_slice::<PduEvent>(&v)
                    .map_err(|_| Error::bad_database("PDU in db is invalid."))?;
                if pdu.sender != user_id {
-                    pdu.remove_transaction_id()?;
+                    pdu.unsigned.remove("transaction_id");
                }
                Ok((pdu_id, pdu))
            }))
@@ -2386,7 +2280,7 @@ impl Rooms {
                let mut pdu = serde_json::from_slice::<PduEvent>(&v)
                    .map_err(|_| Error::bad_database("PDU in db is invalid."))?;
                if pdu.sender != user_id {
-                    pdu.remove_transaction_id()?;
+                    pdu.unsigned.remove("transaction_id");
                }
                Ok((pdu_id, pdu))
            }))
@@ -2455,9 +2349,11 @@ impl Rooms {
                    if let Some(predecessor) = self
                        .room_state_get(room_id, &EventType::RoomCreate, "")?
                        .and_then(|create| {
-                            serde_json::from_str::<ruma::events::room::create::CreateEventContent>(
-                                create.content.get(),
-                            )
+                            serde_json::from_value::<
+                                Raw<ruma::events::room::create::CreateEventContent>,
+                            >(create.content.clone())
+                            .expect("Raw::from_value always works")
+                            .deserialize()
                            .ok()
                        })
                        .and_then(|content| content.predecessor)
@@ -2805,15 +2701,17 @@ impl Rooms {
            );
            let state_lock = mutex_state.lock().await;

-            let mut event = serde_json::from_str::<member::MemberEventContent>(
+            let mut event = serde_json::from_value::<Raw<member::MemberEventContent>>(
                self.room_state_get(room_id, &EventType::RoomMember, &user_id.to_string())?
                    .ok_or(Error::BadRequest(
                        ErrorKind::BadState,
                        "Cannot leave a room you are not a member of.",
                    ))?
                    .content
-                    .get(),
+                    .clone(),
            )
+            .expect("from_value::<Raw<..>> can never fail")
+            .deserialize()
            .map_err(|_| Error::bad_database("Invalid member event in database."))?;

            event.membership = member::MembershipState::Leave;
@@ -2821,10 +2719,8 @@ impl Rooms {
            self.build_and_append_pdu(
                PduBuilder {
                    event_type: EventType::RoomMember,
-                    content: RawValue::from_string(
-                        serde_json::to_string(&event).expect("event is valid, we just created it"),
-                    )
-                    .expect("string is valid"),
+                    content: serde_json::to_value(event)
+                        .expect("event is valid, we just created it"),
                    unsigned: None,
                    state_key: Some(user_id.to_string()),
                    redacts: None,
@@ -84,7 +84,7 @@ pub enum SendingEventType {
 pub struct Sending {
    /// The state for a given state hash.
    pub(super) servername_educount: Arc<dyn Tree>, // EduCount: Count of last EDU sync
-    pub(super) servernameevent_data: Arc<dyn Tree>, // ServernameEvent = (+ / $)SenderKey / ServerName / UserId + PduId / Id (for edus), Data = EDU content
+    pub(super) servernameevent_data: Arc<dyn Tree>, // ServernamEvent = (+ / $)SenderKey / ServerName / UserId + PduId / Id (for edus), Data = EDU content
    pub(super) servercurrentevent_data: Arc<dyn Tree>, // ServerCurrentEvents = (+ / $)ServerName / UserId + PduId / Id (for edus), Data = EDU content
    pub(super) maximum_requests: Arc<Semaphore>,
    pub sender: mpsc::UnboundedSender<(Vec<u8>, Vec<u8>)>,
@@ -423,23 +423,13 @@ impl Sending {
        Ok(())
    }

-    #[tracing::instrument(skip(self, servers, pdu_id))]
-    pub fn send_pdu<I: Iterator<Item = Box<ServerName>>>(
-        &self,
-        servers: I,
-        pdu_id: &[u8],
-    ) -> Result<()> {
-        let mut batch = servers.map(|server| {
+    #[tracing::instrument(skip(self, server, pdu_id))]
+    pub fn send_pdu(&self, server: &ServerName, pdu_id: &[u8]) -> Result<()> {
        let mut key = server.as_bytes().to_vec();
        key.push(0xff);
        key.extend_from_slice(pdu_id);
-
-            self.sender.unbounded_send((key.clone(), vec![])).unwrap();
-
-            (key, Vec::new())
-        });
-
-        self.servernameevent_data.insert_batch(&mut batch)?;
+        self.servernameevent_data.insert(&key, &[])?;
+        self.sender.unbounded_send((key, vec![])).unwrap();

        Ok(())
    }
@@ -573,15 +563,9 @@ impl Sending {

                for pdu in pdus {
                    // Redacted events are not notification targets (we don't send push for them)
-                    if let Some(unsigned) = &pdu.unsigned {
-                        if let Ok(unsigned) =
-                            serde_json::from_str::<serde_json::Value>(unsigned.get())
-                        {
-                            if unsigned.get("redacted_because").is_some() {
+                    if pdu.unsigned.get("redacted_because").is_some() {
                        continue;
                    }
-                        }
-                    }

                    let userid =
                        UserId::try_from(utils::string_from_bytes(user).map_err(|_| {
@@ -6,15 +6,15 @@ use ruma::{
        AnySyncStateEvent, EventType, StateEvent,
    },
    serde::{CanonicalJsonObject, CanonicalJsonValue, Raw},
-    state_res, EventId, MilliSecondsSinceUnixEpoch, RoomId, RoomVersionId, ServerName, UInt,
-    UserId,
+    state_res, EventId, MilliSecondsSinceUnixEpoch, RoomId, RoomVersionId, ServerName,
+    ServerSigningKeyId, UInt, UserId,
 };
 use serde::{Deserialize, Serialize};
-use serde_json::{json, value::RawValue};
-use std::{cmp::Ordering, collections::BTreeMap, convert::TryFrom, sync::RwLock};
+use serde_json::json;
+use std::{cmp::Ordering, collections::BTreeMap, convert::TryFrom};
 use tracing::warn;

-#[derive(Deserialize, Serialize, Debug)]
+#[derive(Clone, Deserialize, Serialize, Debug)]
 pub struct PduEvent {
    pub event_id: EventId,
    pub room_id: RoomId,
@@ -22,9 +22,7 @@ pub struct PduEvent {
    pub origin_server_ts: UInt,
    #[serde(rename = "type")]
    pub kind: EventType,
-    pub content: Box<serde_json::value::RawValue>,
-    #[serde(skip)]
-    pub parsed_content: RwLock<Option<serde_json::Value>>,
+    pub content: serde_json::Value,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub state_key: Option<String>,
    pub prev_events: Vec<EventId>,
@@ -32,17 +30,16 @@ pub struct PduEvent {
    pub auth_events: Vec<EventId>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub redacts: Option<EventId>,
-    #[serde(default, skip_serializing_if = "Option::is_none")]
-    pub unsigned: Option<Box<serde_json::value::RawValue>>,
+    #[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
+    pub unsigned: BTreeMap<String, serde_json::Value>,
    pub hashes: EventHash,
-    #[serde(default, skip_serializing_if = "Option::is_none")]
-    pub signatures: Option<Box<serde_json::value::RawValue>>, // BTreeMap<Box<ServerName>, BTreeMap<ServerSigningKeyId, String>>
+    pub signatures: BTreeMap<Box<ServerName>, BTreeMap<ServerSigningKeyId, String>>,
 }

 impl PduEvent {
    #[tracing::instrument(skip(self))]
    pub fn redact(&mut self, reason: &PduEvent) -> crate::Result<()> {
-        self.unsigned = None;
+        self.unsigned.clear();

        let allowed: &[&str] = match self.kind {
            EventType::RoomMember => &["membership"],
@@ -62,9 +59,10 @@ impl PduEvent {
            _ => &[],
        };

-        let mut old_content =
-            serde_json::from_str::<BTreeMap<String, serde_json::Value>>(self.content.get())
-                .map_err(|_| Error::bad_database("PDU in db has invalid content."))?;
+        let old_content = self
+            .content
+            .as_object_mut()
+            .ok_or_else(|| Error::bad_database("PDU in db has invalid content."))?;

        let mut new_content = serde_json::Map::new();

@@ -74,29 +72,12 @@ impl PduEvent {
            }
        }

-        self.unsigned = Some(RawValue::from_string(serde_json::to_string(&json!({
-            "redacted_because": serde_json::to_value(reason).expect("to_value(PduEvent) always works")
-        })).expect("to string always works")).expect("string is valid"));
-
-        self.content = RawValue::from_string(
-            serde_json::to_string(&new_content).expect("to string always works"),
-        )
-        .expect("string is valid");
-
-        Ok(())
-    }
-
-    pub fn remove_transaction_id(&mut self) -> crate::Result<()> {
-        if let Some(unsigned) = &self.unsigned {
-            let mut unsigned =
-                serde_json::from_str::<BTreeMap<String, Box<RawValue>>>(unsigned.get())
-                    .map_err(|_| Error::bad_database("Invalid unsigned in pdu event"))?;
-            unsigned.remove("transaction_id");
-            self.unsigned = Some(
-                RawValue::from_string(serde_json::to_string(&unsigned).expect("unsigned is valid"))
-                    .expect("string is valid"),
+        self.unsigned.insert(
+            "redacted_because".to_owned(),
+            serde_json::to_value(reason).expect("to_value(PduEvent) always works"),
        );
-        }
+
+        self.content = new_content.into();

        Ok(())
    }
@@ -284,14 +265,8 @@ impl state_res::Event for PduEvent {
        &self.kind
    }

-    fn content(&self) -> serde_json::Value {
-        self.parsed_content
-            .write()
-            .unwrap()
-            .get_or_insert_with(|| {
-                serde_json::to_value(&self.content).expect("content is valid json")
-            })
-            .clone()
+    fn content(&self) -> &serde_json::Value {
+        &self.content
    }

    fn origin_server_ts(&self) -> MilliSecondsSinceUnixEpoch {
@@ -323,17 +298,11 @@ impl state_res::Event for PduEvent {
    }

    fn signatures(&self) -> BTreeMap<Box<ServerName>, BTreeMap<ruma::ServerSigningKeyId, String>> {
-        self.signatures
-            .as_ref()
-            .map(|raw| serde_json::from_str(raw.get()).expect("string is valid signatures json"))
-            .unwrap_or_default()
+        self.signatures.clone()
    }

-    fn unsigned(&self) -> BTreeMap<String, serde_json::Value> {
-        self.unsigned
-            .as_ref()
-            .map(|raw| serde_json::from_str(raw.get()).expect("string is valid json"))
-            .unwrap_or_default()
+    fn unsigned(&self) -> &BTreeMap<String, serde_json::Value> {
+        &self.unsigned
    }
 }

@@ -383,7 +352,7 @@ pub(crate) fn gen_event_id_canonical_json(
 pub struct PduBuilder {
    #[serde(rename = "type")]
    pub event_type: EventType,
-    pub content: Box<serde_json::value::RawValue>,
+    pub content: serde_json::Value,
    pub unsigned: Option<BTreeMap<String, serde_json::Value>>,
    pub state_key: Option<String>,
    pub redacts: Option<EventId>,
@@ -394,12 +363,8 @@ impl From<AnyInitialStateEvent> for PduBuilder {
    fn from(event: AnyInitialStateEvent) -> Self {
        Self {
            event_type: EventType::from(event.event_type()),
-            content: RawValue::from_string(
-                serde_json::to_string(&event.content()).expect(
-                    "AnyStateEventContent came from JSON and can thus turn back into JSON.",
-                ),
-            )
-            .expect("string is valid"),
+            content: serde_json::value::to_value(event.content())
+                .expect("AnyStateEventContent came from JSON and can thus turn back into JSON."),
            unsigned: None,
            state_key: Some(event.state_key().to_owned()),
            redacts: None,
@@ -39,6 +39,7 @@ use ruma::{
    },
    directory::{IncomingFilter, IncomingRoomNetwork},
    events::{
+        pdu::Pdu,
        receipt::{ReceiptEvent, ReceiptEventContent},
        room::{
            create::CreateEventContent,
@@ -54,9 +55,7 @@ use ruma::{
    uint, EventId, MilliSecondsSinceUnixEpoch, RoomId, RoomVersionId, ServerName,
    ServerSigningKeyId,
 };
-use serde_json::value::RawValue;
 use std::{
-    cell::RefCell,
    collections::{btree_map, hash_map, BTreeMap, BTreeSet, HashMap, HashSet},
    convert::{TryFrom, TryInto},
    fmt::Debug,
@@ -1096,7 +1095,7 @@ pub(crate) async fn handle_incoming_pdu<'a>(
            let start_time = Instant::now();
            let event_id = pdu.event_id.clone();
            if let Err(e) = upgrade_outlier_to_timeline_pdu(
-                &pdu,
+                pdu,
                json,
                &create_event,
                origin,
@@ -1120,7 +1119,7 @@ pub(crate) async fn handle_incoming_pdu<'a>(
    }

    upgrade_outlier_to_timeline_pdu(
-        &incoming_pdu,
+        incoming_pdu,
        val,
        &create_event,
        origin,
@@ -1140,7 +1139,8 @@ fn handle_outlier_pdu<'a>(
    value: BTreeMap<String, CanonicalJsonValue>,
    db: &'a Database,
    pub_key_map: &'a RwLock<BTreeMap<String, BTreeMap<String, String>>>,
-) -> AsyncRecursiveType<'a, StdResult<(PduEvent, BTreeMap<String, CanonicalJsonValue>), String>> {
+) -> AsyncRecursiveType<'a, StdResult<(Arc<PduEvent>, BTreeMap<String, CanonicalJsonValue>), String>>
+{
    Box::pin(async move {
        // TODO: For RoomVersion6 we must check that Raw<..> is canonical do we anywhere?: https://matrix.org/docs/spec/rooms/v6#canonical-json

@@ -1153,9 +1153,10 @@ fn handle_outlier_pdu<'a>(
        // 2. Check signatures, otherwise drop
        // 3. check content hash, redact if doesn't match

-        let create_event_content = serde_json::from_str::<CreateEventContent>(
-            create_event.content.get(),
-        )
+        let create_event_content =
+            serde_json::from_value::<Raw<CreateEventContent>>(create_event.content.clone())
+                .expect("Raw::from_value always works.")
+                .deserialize()
                .map_err(|e| {
                    warn!("Invalid create event: {}", e);
                    "Invalid create event in db.".to_owned()
@@ -1240,7 +1241,7 @@ fn handle_outlier_pdu<'a>(
                    .expect("all auth events have state keys"),
            )) {
                hash_map::Entry::Vacant(v) => {
-                    v.insert(auth_event);
+                    v.insert(auth_event.clone());
                }
                hash_map::Entry::Occupied(_) => {
                    return Err(
@@ -1252,7 +1253,11 @@ fn handle_outlier_pdu<'a>(
        }

        // The original create event must be in the auth events
-        if auth_events.get(&(EventType::RoomCreate, "".to_owned())) != Some(create_event) {
+        if auth_events
+            .get(&(EventType::RoomCreate, "".to_owned()))
+            .map(|a| a.as_ref())
+            != Some(create_event)
+        {
            return Err("Incoming event refers to wrong create event.".to_owned());
        }

@@ -1263,7 +1268,7 @@ fn handle_outlier_pdu<'a>(
            db.rooms
                .get_pdu(&incoming_pdu.auth_events[0])
                .map_err(|e| e.to_string())?
-                .filter(|maybe_create| maybe_create == create_event)
+                .filter(|maybe_create| **maybe_create == *create_event)
        } else {
            None
        };
@@ -1271,7 +1276,7 @@ fn handle_outlier_pdu<'a>(
        if !state_res::event_auth::auth_check(
            &room_version,
            &incoming_pdu,
-            previous_create.as_ref(),
+            previous_create,
            None::<PduEvent>, // TODO: third party invite
            |k, s| auth_events.get(&(k.clone(), s.to_owned())),
        )
@@ -1288,13 +1293,13 @@ fn handle_outlier_pdu<'a>(
            .map_err(|_| "Failed to add pdu as outlier.".to_owned())?;
        debug!("Added pdu as outlier.");

-        Ok((incoming_pdu, val))
+        Ok((Arc::new(incoming_pdu), val))
    })
 }

 #[tracing::instrument(skip(incoming_pdu, val, create_event, origin, db, room_id, pub_key_map))]
 async fn upgrade_outlier_to_timeline_pdu(
-    incoming_pdu: &PduEvent,
+    incoming_pdu: Arc<PduEvent>,
    val: BTreeMap<String, CanonicalJsonValue>,
    create_event: &PduEvent,
    origin: &ServerName,
@@ -1315,7 +1320,10 @@ async fn upgrade_outlier_to_timeline_pdu(
    }

    let create_event_content =
-        serde_json::from_str::<CreateEventContent>(create_event.content.get()).map_err(|e| {
+        serde_json::from_value::<Raw<CreateEventContent>>(create_event.content.clone())
+            .expect("Raw::from_value always works.")
+            .deserialize()
+            .map_err(|e| {
                warn!("Invalid create event: {}", e);
                "Invalid create event in db.".to_owned()
            })?;
@@ -1426,20 +1434,16 @@ async fn upgrade_outlier_to_timeline_pdu(
                fork_states.push(state);
            }

-            let event_cache = RefCell::new(HashMap::new());
            state_at_incoming_event = match state_res::resolve(
                room_version_id,
                &fork_states,
                auth_chain_sets,
-                |id| match event_cache.borrow_mut().entry(id.clone()) {
-                    hash_map::Entry::Vacant(v) => {
+                |id| {
                    let res = db.rooms.get_pdu(id);
                    if let Err(e) = &res {
                        error!("LOOK AT ME Failed to fetch event: {}", e);
                    }
-                        v.insert(res.ok().flatten().map(Arc::new)).clone()
-                    }
-                    hash_map::Entry::Occupied(o) => o.get().as_ref().map(Arc::clone),
+                    res.ok().flatten()
                },
            ) {
                Ok(new_state) => Some(
@@ -1550,7 +1554,7 @@ async fn upgrade_outlier_to_timeline_pdu(
        db.rooms
            .get_pdu(&incoming_pdu.auth_events[0])
            .map_err(|e| e.to_string())?
-            .filter(|maybe_create| maybe_create == create_event)
+            .filter(|maybe_create| **maybe_create == *create_event)
    } else {
        None
    };
@@ -1558,7 +1562,7 @@ async fn upgrade_outlier_to_timeline_pdu(
    let check_result = state_res::event_auth::auth_check(
        &room_version,
        &incoming_pdu,
-        previous_create.as_ref(),
+        previous_create.as_deref(),
        None::<PduEvent>, // TODO: third party invite
        |k, s| {
            db.rooms
@@ -1642,7 +1646,7 @@ async fn upgrade_outlier_to_timeline_pdu(
    let soft_fail = !state_res::event_auth::auth_check(
        &room_version,
        &incoming_pdu,
-        previous_create.as_ref(),
+        previous_create.as_deref(),
        None::<PduEvent>,
        |k, s| auth_events.get(&(k.clone(), s.to_owned())),
    )
@@ -1787,20 +1791,16 @@ async fn upgrade_outlier_to_timeline_pdu(
                .collect::<Result<Vec<_>>>()
                .map_err(|_| "Failed to get_statekey_from_short.".to_owned())?;

-            let event_cache = RefCell::new(HashMap::new());
            let state = match state_res::resolve(
                room_version_id,
                fork_states,
                auth_chain_sets,
-                |id| match event_cache.borrow_mut().entry(id.clone()) {
-                    hash_map::Entry::Vacant(v) => {
+                |id| {
                    let res = db.rooms.get_pdu(id);
                    if let Err(e) = &res {
                        error!("LOOK AT ME Failed to fetch event: {}", e);
                    }
-                        v.insert(res.ok().flatten().map(Arc::new)).clone()
-                    }
-                    hash_map::Entry::Occupied(o) => o.get().as_ref().map(Arc::clone),
+                    res.ok().flatten()
                },
            ) {
                Ok(new_state) => new_state,
@@ -1873,7 +1873,7 @@ pub(crate) fn fetch_and_handle_outliers<'a>(
    create_event: &'a PduEvent,
    room_id: &'a RoomId,
    pub_key_map: &'a RwLock<BTreeMap<String, BTreeMap<String, String>>>,
-) -> AsyncRecursiveType<'a, Vec<(PduEvent, Option<BTreeMap<String, CanonicalJsonValue>>)>> {
+) -> AsyncRecursiveType<'a, Vec<(Arc<PduEvent>, Option<BTreeMap<String, CanonicalJsonValue>>)>> {
    Box::pin(async move {
        let back_off = |id| match db.globals.bad_event_ratelimiter.write().unwrap().entry(id) {
            hash_map::Entry::Vacant(e) => {
@@ -2669,7 +2669,10 @@ pub fn create_join_event_template_route(
    let create_event_content = create_event
        .as_ref()
        .map(|create_event| {
-            serde_json::from_str::<CreateEventContent>(create_event.content.get()).map_err(|e| {
+            serde_json::from_value::<Raw<CreateEventContent>>(create_event.content.clone())
+                .expect("Raw::from_value always works.")
+                .deserialize()
+                .map_err(|e| {
                    warn!("Invalid create event: {}", e);
                    Error::bad_database("Invalid create event in db.")
                })
@@ -2699,8 +2702,7 @@ pub fn create_join_event_template_route(
        ));
    }

-    let content = RawValue::from_string(
-        serde_json::to_string(&MemberEventContent {
+    let content = serde_json::to_value(MemberEventContent {
        avatar_url: None,
        blurhash: None,
        displayname: None,
@@ -2709,9 +2711,7 @@ pub fn create_join_event_template_route(
        third_party_invite: None,
        reason: None,
    })
-        .expect("member event is valid value"),
-    )
-    .expect("string is valid");
+    .expect("member event is valid value");

    let state_key = body.user_id.to_string();
    let kind = EventType::RoomMember;
@@ -2738,7 +2738,7 @@ pub fn create_join_event_template_route(
        unsigned.insert("prev_content".to_owned(), prev_pdu.content.clone());
        unsigned.insert(
            "prev_sender".to_owned(),
-            serde_json::from_str(prev_pdu.sender.as_str()).expect("UserId is valid string"),
+            serde_json::to_value(&prev_pdu.sender).expect("UserId::to_value always works"),
        );
    }

@@ -2751,7 +2751,6 @@ pub fn create_join_event_template_route(
            .expect("time is valid"),
        kind,
        content,
-        parsed_content: RwLock::new(None),
        state_key: Some(state_key),
        prev_events,
        depth,
@@ -2760,26 +2759,17 @@ pub fn create_join_event_template_route(
            .map(|(_, pdu)| pdu.event_id.clone())
            .collect(),
        redacts: None,
-        unsigned: if unsigned.is_empty() {
-            None
-        } else {
-            Some(
-                RawValue::from_string(
-                    serde_json::to_string(&unsigned).expect("to_string always works"),
-                )
-                .expect("string is valid"),
-            )
-        },
+        unsigned,
        hashes: ruma::events::pdu::EventHash {
            sha256: "aaa".to_owned(),
        },
-        signatures: None,
+        signatures: BTreeMap::new(),
    };

    let auth_check = state_res::auth_check(
        &room_version,
        &pdu,
-        create_prev_event,
+        create_prev_event.as_deref(),
        None::<PduEvent>, // TODO: third_party_invite
        |k, s| auth_events.get(&(k.clone(), s.to_owned())),
    )
@@ -2890,13 +2880,14 @@ async fn create_join_event(
        db,
    )?;

-    let servers = db
+    for server in db
        .rooms
        .room_servers(room_id)
        .filter_map(|r| r.ok())
-        .filter(|server| &**server != db.globals.server_name());
-
-    db.sending.send_pdu(servers, &pdu_id)?;
+        .filter(|server| &**server != db.globals.server_name())
+    {
+        db.sending.send_pdu(&server, &pdu_id)?;
+    }

    db.flush()?;

@@ -3289,13 +3280,45 @@ pub(crate) async fn fetch_required_signing_keys(

 // Gets a list of servers for which we don't have the signing key yet. We go over
 // the PDUs and either cache the key or add it to the list that needs to be retrieved.
-pub fn get_server_keys_from_cache(
-    pdu: &CanonicalJsonObject,
+fn get_server_keys_from_cache(
+    pdu: &Raw<Pdu>,
    servers: &mut BTreeMap<Box<ServerName>, BTreeMap<ServerSigningKeyId, QueryCriteria>>,
+    room_version: &RoomVersionId,
    pub_key_map: &mut RwLockWriteGuard<'_, BTreeMap<String, BTreeMap<String, String>>>,
    db: &Database,
 ) -> Result<()> {
-    let signatures = pdu
+    let value = serde_json::from_str::<CanonicalJsonObject>(pdu.json().get()).map_err(|e| {
+        error!("Invalid PDU in server response: {:?}: {:?}", pdu, e);
+        Error::BadServerResponse("Invalid PDU in server response")
+    })?;
+
+    let event_id = EventId::try_from(&*format!(
+        "${}",
+        ruma::signatures::reference_hash(&value, room_version)
+            .expect("ruma can calculate reference hashes")
+    ))
+    .expect("ruma's reference hashes are valid event ids");
+
+    if let Some((time, tries)) = db
+        .globals
+        .bad_event_ratelimiter
+        .read()
+        .unwrap()
+        .get(&event_id)
+    {
+        // Exponential backoff
+        let mut min_elapsed_duration = Duration::from_secs(30) * (*tries) * (*tries);
+        if min_elapsed_duration > Duration::from_secs(60 * 60 * 24) {
+            min_elapsed_duration = Duration::from_secs(60 * 60 * 24);
+        }
+
+        if time.elapsed() < min_elapsed_duration {
+            debug!("Backing off from {}", event_id);
+            return Err(Error::BadServerResponse("bad event, still backing off"));
+        }
+    }
+
+    let signatures = value
        .get("signatures")
        .ok_or(Error::BadServerResponse(
            "No signatures in server response pdu.",
@@ -3347,11 +3370,30 @@ pub fn get_server_keys_from_cache(
 }

 pub(crate) async fn fetch_join_signing_keys(
-    servers: Arc<RwLock<BTreeMap<Box<ServerName>, BTreeMap<ServerSigningKeyId, QueryCriteria>>>>,
+    event: &create_join_event::v2::Response,
+    room_version: &RoomVersionId,
    pub_key_map: &RwLock<BTreeMap<String, BTreeMap<String, String>>>,
    db: &Database,
 ) -> Result<()> {
-    let mut servers = std::mem::take(&mut *servers.write().unwrap());
+    let mut servers =
+        BTreeMap::<Box<ServerName>, BTreeMap<ServerSigningKeyId, QueryCriteria>>::new();
+
+    {
+        let mut pkm = pub_key_map
+            .write()
+            .map_err(|_| Error::bad_database("RwLock is poisoned."))?;
+
+        // Try to fetch keys, failure is okay
+        // Servers we couldn't find in the cache will be added to `servers`
+        for pdu in &event.room_state.state {
+            let _ = get_server_keys_from_cache(pdu, &mut servers, room_version, &mut pkm, db);
+        }
+        for pdu in &event.room_state.auth_chain {
+            let _ = get_server_keys_from_cache(pdu, &mut servers, room_version, &mut pkm, db);
+        }
+
+        drop(pkm);
+    }

    if servers.is_empty() {
        // We had all keys locally
Author	SHA1	Message	Date
Jonas Zohren	d8b3003273	docs: WIP: Adjust suggested value ranges to Timo's suggestion and when to change them Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-09-15 12:45:13 +02:00
Jonas Zohren	edd690c583	docs: WIP: Added some suggested range for numeric values Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-09-15 12:38:17 +02:00
Jonas Zohren	c2068a434e	docs: WIP: More config options Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-09-15 12:21:43 +02:00
Jonas Zohren	b981d3d039	Move some documentation to docs/ and start overhaul	2021-09-15 11:16:19 +02:00
Jonas Zohren	80a2711f45	Move some documentation to docs/ and start overhaul	2021-09-15 11:16:19 +02:00
Jonas Zohren	2f4885c2a1	Remove the "register an account with element" test Broke due to a timeout and Timo does not like broken tests. Less testing means less failing tests. Also, hopefully sytest is less broken now.	2021-09-15 11:16:18 +02:00
Timo Kösters	3fec75a7b6	improvement: more efficient sqlite	2021-09-15 11:16:18 +02:00
Timo Kösters	5094f8e7f1	improvement: deduplicate watchers	2021-09-15 11:16:18 +02:00
Jonas Platte	cd19f02d3b	Clean up (mostly automated with cargo clippy --fix)	2021-09-15 11:16:18 +02:00
Jonas Platte	dbfe16bdb2	Upgrade ruma	2021-09-15 11:16:18 +02:00
Jonas Platte	5bdeb971d8	Get rid of more unnecessary intermediate collections	2021-09-15 11:16:18 +02:00
Daniel Wiesenberg	a38b23d56d	Add mautrix-signal appservice instructions	2021-09-15 11:16:18 +02:00
Jonathan de Jong	25a81c480b	fix sync not firing on new events in room	2021-09-15 11:16:16 +02:00
Rasmus Thomsen	1282405f1d	fix(database): handle errors in config parsin or database creation Showing the user a backtrace can be pretty confusing, so just printing a nice error message makes errors easier to understand for end users. fixes #121	2021-09-15 11:16:16 +02:00
Jonas Platte	ca4b676b4c	Remove unused dependencies	2021-09-15 11:16:16 +02:00
charludo	3db558b2f8	These lines get generated by certbot. Having them in the file before running certbot results in an apache2 error, and putting them in afterwards is not necessary, since certbot places them there on its own.	2021-09-15 11:16:16 +02:00
Daniel Wiesenberg	5ea10bbd8e	Fix deprecated/removed Traefik label	2021-09-15 11:16:16 +02:00
Timo Kösters	31b2728551	fix: make sure old events don't sneek into the timeline	2021-09-15 11:16:16 +02:00
Jonas Zohren	d48b96fda6	chore(CI): Adjust CI for master and next branch development model - Build release builds for branches "master" and "next" - Push docker images under different tags, depending on why the pipeline started - branch master: push to `latest` - branch next: push to `next` - tag: push to `$TAG_NAME` Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-09-15 11:16:16 +02:00
Timo Kösters	fe83c80a65	docs: add thanks to	2021-09-15 11:16:16 +02:00
Jonas Zohren	98ce5f62d5	chore: Also run CI on git tags, not only new commits. Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-09-15 11:16:16 +02:00
Timo Kösters	870c178c8b	fix: avoid panic	2021-09-15 11:16:16 +02:00
Timo Kösters	1b9f56e48c	chore: update dependencies	2021-09-15 11:16:16 +02:00
Timo Kösters	3143d468c7	revert Arc<EventId> commit	2021-09-15 11:16:15 +02:00
Timo Kösters	ac594f5502	docs: update readme and version	2021-09-15 11:16:15 +02:00
Jonas Platte	75701f75e4	Remove unneeded intermediary container collection	2021-09-15 11:16:09 +02:00
Timo Kösters	186722c51f	fix: batch key fetching	2021-09-15 11:16:09 +02:00
Kurt Roeckx	fc3ff86ad9	fixup! Get required keys in batch when joining a room	2021-09-15 11:16:09 +02:00
Kurt Roeckx	a8d181e00e	fixup! Get required keys in batch when joining a room	2021-09-15 11:16:09 +02:00
Kurt Roeckx	61adef8f2c	Let our server signing be valid for 1 week It was valid for 2 minutes, which can result in the server being asked too much for something that never changes.	2021-09-15 11:16:09 +02:00
Kurt Roeckx	13c9f2fb50	Get required keys in batch when joining a room We now ask the trusted server for all keys in 1 request, instead of asking each server individual for it's own keys.	2021-09-15 11:16:09 +02:00
Daniel Wiesenberg	5d279159e3	Add Traefik setup, incl. step-by-step, to docker README.	2021-09-15 11:16:09 +02:00
Jonas Platte	e14db60b6c	Fix a bunch of clippy lints	2021-09-15 11:16:09 +02:00
Timo Kösters	4cba464faf	improvement: make pdu cache capacity configurable	2021-09-15 11:16:09 +02:00
Timo Kösters	828611c0e4	improvement: stop prev event fetching if too many events fail	2021-09-15 11:16:09 +02:00
Timo Kösters	e206b0ca8f	docs: documentation for every endpoint	2021-09-15 11:16:09 +02:00
Daniel Wiesenberg	4c971d7695	Fix healthcheck.sh permissions and rearange ci dockerfile	2021-09-15 11:16:09 +02:00
Jonathan de Jong	8aed9b7711	registration default true	2021-09-15 11:16:09 +02:00
Daniel Wiesenberg	3f766d2a13	Use `$CI_COMMIT_SHORT_SHA` for `GIT_REF` Using `$CI_COMMIT_REF_NAME` means we get `master` for every image build, which is not very useful/informative. Using `$CI_COMMIT_SHORT_SHA`, on the other hand, makes it possible to see exactly from which commit an image was built.	2021-09-15 11:16:09 +02:00
Daniel Wiesenberg	6d6ce0f697	Change healthcheck in ci dockerfile	2021-09-15 11:16:09 +02:00
Daniel Wiesenberg	8b4389e0c2	Move docker healthcheck into dedicated script.	2021-09-15 11:16:09 +02:00
Jonathan de Jong	fc29ad088c	add tls	2021-09-15 11:16:09 +02:00
Timo Kösters	b0aa4fc7f5	improvement: call /state_ids less often by using state res	2021-09-15 11:16:09 +02:00
Timo Kösters	73ae29c0de	fix: inviting dendrite users	2021-09-15 11:16:09 +02:00
Timo Kösters	8c7ae60518	improvement: don't do state updates if the event was soft failed	2021-09-15 11:16:09 +02:00
Timo Kösters	4d6af6ee88	fix: make appservices more efficient	2021-09-15 11:16:09 +02:00
Timo Kösters	4ede0dc2d9	add warning if calculated event id != requested event id	2021-09-15 11:16:09 +02:00
Timo Kösters	4ce02e8ff6	fix: don't retry soft failed events	2021-09-15 11:16:09 +02:00
Devin Ragotzy	4ac40dec9a	Use Arc<EventId> in place of most EventIds	2021-09-15 11:16:09 +02:00
Timo Kösters	51b96b3538	fix: server resolution	2021-09-15 11:16:09 +02:00
Timo Kösters	5b904fbef4	fix: server resolution with well-known files	2021-09-15 11:16:09 +02:00
Timo Kösters	6c0079f2c4	improvement: less IO for auth chains	2021-09-15 11:16:09 +02:00
Timo Kösters	3a588c4561	fix: improve key fetching	2021-09-15 11:16:09 +02:00
Timo Kösters	f22ad5dfba	chore: bump dependencies	2021-09-15 11:16:09 +02:00
Timo Kösters	2b50afcc0a	improvement: persist cached auth chains in db	2021-09-15 11:16:09 +02:00
Timo Kösters	8216abc8c5	fix: better migration, force e2ee device key updates	2021-09-15 11:16:09 +02:00
Timo Kösters	981d054b0c	fix: wrong soft fail check, too many events in /sync state response	2021-09-15 11:16:09 +02:00
Timo Kösters	72e2b643bb	fix: e2ee over federation to device events were not being sent	2021-09-15 11:16:09 +02:00
Timo Kösters	ee4b08c185	improvement: more efficient auth chain cache	2021-09-15 11:16:09 +02:00
Timo Kösters	896427b67f	improvement: better e2ee over fed, faster incoming event handling	2021-09-15 11:16:09 +02:00
Jonas Zohren	95989241db	Only apply max. optimizations in CI builds. The average german man has a life expectancy of 78.7 years, or 689884.2 hours. Assuming that Timo is 20 years old, he has rougly 514564.2 hours left on planet earth. Also assuming that cross release builds took him 25 minutes before, but 2-2.5x of that with the current release compilation config he wasted roughly an hour waiting for it to complete. If he continued to work on Conduit for 20 more years (or 175320 hours), and makes a release compilation about once per day, this means 7305 hours or 304 days wasted waiting for the rust compiler. By cutting that back down to the original settings, he get's 182 days of his life back. That's about 0.63% of his remaining life. 182 joyful days he can spend with family and loved ones.	2021-09-15 11:16:09 +02:00
Timo Kösters	10da08e260	fix: room upgrades	2021-09-15 11:16:09 +02:00
Timo Kösters	6255406de0	fix: correct create event warnings	2021-09-15 11:16:09 +02:00
Kurt Roeckx	4c44a7adf5	Implement federation/v1/send_join	2021-09-15 11:16:09 +02:00
Tom Smeding	48fb8c5d4d	Consistent and escaped response in get_pdu 1. The fallback text of the get_pdu admin room command response message now contains the same text as the formatted_body content (namely, the json instead of Debug-formatting of a serde type). 2. The formatted_body content of the get_pdu response is now html-escaped.	2021-09-15 11:16:09 +02:00
Timo Kösters	d058f6522e	fmt	2021-09-15 11:16:09 +02:00
Timo Kösters	8784df0d3a	improvement: limit prev event fetching	2021-09-15 11:16:09 +02:00
Timo Kösters	524b4960cd	fix: don't save empty tokens	2021-09-15 11:16:09 +02:00
Timo Kösters	0176d40347	improvement: faster incoming transaction handling	2021-09-15 11:16:09 +02:00
Timo Kösters	b74b8c48ce	improvement: better prev event fetching, perf improvements	2021-09-15 11:16:09 +02:00
Timo Kösters	8a4b7cf4bd	fix: faster room joins	2021-09-15 11:16:09 +02:00
Timo Kösters	4a9483c07c	fix: don't load endless prev events and fix room join bug	2021-09-15 11:16:09 +02:00
Timo Kösters	8f1a41dcd2	improvement: better sqlite	2021-09-15 11:16:09 +02:00
Timo Kösters	6f58af0b99	improvement: state info cache	2021-09-15 11:16:09 +02:00
Timo Kösters	ac9dce2bdf	fix: fetch event multiple times	2021-09-15 11:16:09 +02:00
Timo Kösters	197a111748	fix: also fetch prev events that are outliers already	2021-09-15 11:16:09 +02:00
Timo Kösters	b9f138450c	fix: fetch more than one prev event	2021-09-15 11:16:09 +02:00
Timo Kösters	1aecadf990	fix: don't use recursion for prev events	2021-09-15 11:16:09 +02:00
Timo Kösters	7514aa5d95	fmt	2021-09-15 11:16:09 +02:00
Timo Kösters	c0c783fdec	remove prev event fetch limit	2021-09-15 11:16:09 +02:00
Timo Kösters	b02e4bf46c	fix: delta calculation	2021-09-15 11:16:09 +02:00
Timo Kösters	87f5f17134	fix: room exists panic	2021-09-15 11:16:09 +02:00
Timo Kösters	f1aea98384	finish implementing better state store	2021-09-15 11:16:09 +02:00
Timo Kösters	caebb13a98	improvement: migrations, batch inserts	2021-09-15 11:16:09 +02:00
Timo Kösters	0b073c6534	WIP improvement: much better state storage	2021-09-15 11:16:09 +02:00
Timo Kösters	f29a6d7945	fix: long prev event fetch times for huge rooms	2021-09-15 11:16:09 +02:00
Timo Kösters	8696e627b0	less warnings	2021-09-15 11:16:09 +02:00
Timo Kösters	201de77db4	improvement: cache for short event ids	2021-09-15 11:16:09 +02:00
Timo Kösters	33e84c74ac	improvement: use u64s in auth chain cache	2021-09-15 11:16:09 +02:00
Timo Kösters	39d1f86050	improvement: smaller cache, better prev event fetching	2021-09-15 11:16:09 +02:00
Jonas Zohren	afa71756c0	WIP: change layout of config options documentation	2021-09-07 11:36:28 +02:00
Jonas Zohren	d0e06b49da	Switch to mdbook Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-08-19 16:46:03 +02:00
Jonas Zohren	5547c0f579	Merge remote-tracking branch 'origin/docs-overhaul' into docs-overhaul # Conflicts: # mkdocs.yml	2021-08-19 12:37:05 +02:00
Jonas Zohren	66bc3958ff	Document configuration options in own file Signed-off-by: Jonas Zohren <git-pbkyr@jzohren.de>	2021-08-19 12:36:29 +02:00
Jonas Zohren	7e48567d4b	Move some documentation to docs/ and start overhaul	2021-08-19 11:39:30 +02:00
Jonas Zohren	10ee0a53f9	Move some documentation to docs/ and start overhaul	2021-07-23 16:56:16 +02:00