!DeRvyHqkqkIBbBtwsO:matrix.org

Homeserver Developers

378 Members
If you are building a homeserver, or you want to talk to other people that build homeservers, then this is the place for you! 🥳143 Servers

Load older messages


SenderMessageTime
19 Sep 2024
@gnu_ponut:matrix.orgGnuxie 💜🐝
In reply to @xiretza:xiretza.xyz

is there any place in the spec that tells you to redact things in an attempt to fix invalid data? where is this idea coming from?

Someone said earlier that this is where servers should do
09:27:45
@deepbluev7:neko.devNotHSDevNicoOr at least it has to yield the same result09:27:53
@gnu_ponut:matrix.orgGnuxie 💜🐝* Someone said earlier that this is what servers should do.09:28:22
@neilalexander:dendrite.matrix.orgneilalexander
In reply to @xiretza:xiretza.xyz

is there any place in the spec that tells you to redact things in an attempt to fix invalid data? where is this idea coming from?

The spec doesn't say to do this, no, because it's not safe to do. The hash and signature checks are on the redacted event, but any key that we are supposed to keep could still be invalid but have been hashed/signed anyway
09:28:28
@xiretza:xiretza.xyzxiretza
In reply to @gnu_ponut:matrix.org
Someone said earlier that this is what servers should do.
yes, that's why I'm asking where in the spec this is coming from
09:29:42
@kegan:matrix.orgKegan
In reply to @richvdh:sw1v.org
Isn't this https://github.com/matrix-org/matrix-spec-proposals/pull/2801 writ large?

This MSC doesn't seem to touch on the main thing here. We currently use key names to namespace everything, that's why we have unstable prefixes. It is entirely reasonable to enforce the types of values for known keys. In this case the type was invalid, so it is correctly rejected. This preserves extensibility by ignoring unknown keys

09:31:25
@kegan:matrix.orgKegan

Your point on "server that doesn't know of key X can't validate it so clients need to check anyway" is valid, but for servers this should not be an issue, the validation requirements are different imo

09:32:34
@reivilibre.element:librepush.netOlivier 'reivilibre'is it considered acceptable that an old server that doesn't know about the key yet, would be able to join the room, but a newer one cannot?09:33:41
@reivilibre.element:librepush.netOlivier 'reivilibre'I oscillate in what I think on that09:33:49
@reivilibre.element:librepush.netOlivier 'reivilibre'but at least least, it seems like it would be a minimal effort thing to include a JSON schema of known allowed events in each room version for example (though I guess that wouldn't help how servers have to fall back to ugly handling for older room verisons)09:34:33
@neilalexander:dendrite.matrix.orgneilalexander
In reply to @reivilibre.element:librepush.net
is it considered acceptable that an old server that doesn't know about the key yet, would be able to join the room, but a newer one cannot?
I'm not really sure this matters, a past server might not have known about that key but that's unavoidable, the fact is that servers today can know about it because it's in the spec
09:35:41
@cat:feline.supportCatIsnt room versions literally how we are supposed to break validation compat?09:35:54
@reivilibre.element:librepush.netOlivier 'reivilibre'alternatively, the spec should define semantics in 'layers' — each layer is a JSONSchema that applies to portion of an event, e.g. there would be a 'm.room.create predecessor layer', which defines exactly how to validate the predecessor portion of the event, and every server agrees to only apply the semantic of a room predecessor if that layer validates09:36:03
@reivilibre.element:librepush.netOlivier 'reivilibre'currently you just wind up with one table of what fields and expected types you have on an event, but it's not crystal clear what semantics apply if some of those are valid and others are not09:37:41
@deepbluev7:neko.devNotHSDevNicoAt the very least the client api should validate that known keys of known events have the right type.09:38:05
@reivilibre.element:librepush.netOlivier 'reivilibre'yes, 100% on this, it would be very reasonable to do some JSONSchema validation on new events originating at the server09:38:43
@kegan:matrix.orgKeganI agree, which effectively means once a key name's type is decided it cannot change.09:39:31
@deepbluev7:neko.devNotHSDevNico And if people turn off that validation, it is fair if servers refuse to join those rooms, because nobody needs to respect tulir's experiments! :p 09:39:46
@neilalexander:dendrite.matrix.orgneilalexander
In reply to @reivilibre.element:librepush.net
alternatively, the spec should define semantics in 'layers' — each layer is a JSONSchema that applies to portion of an event, e.g. there would be a 'm.room.create predecessor layer', which defines exactly how to validate the predecessor portion of the event, and every server agrees to only apply the semantic of a room predecessor if that layer validates
Layering like this is also a bit slippery. Take for example the difference in how Synapse decodes JSON (decoding into what is effectively a hashmap and tolerating whatever) vs how Dendrite does (decoding into a typed structure and erroring on type mismatches). Unless we're prescribing that JSON has to be decoded in multiple stages for specific event types, which might not be very feasible in some languages, we're not really getting to the root of the issue. The spec has ultimately told us that certain keys with certain names should be certain types, the bottom line is that is the rule everyone should be playing by
09:44:06
@neilalexander:dendrite.matrix.orgneilalexanderEvery time we grandfather these bugs into the spec, or we add in subtle behaviours to work around them, we make the implementation of the next Matrix homeserver considerably harder, and we introduce more potential for interoperability issues09:45:03
@reivilibre.element:librepush.netOlivier 'reivilibre'
In reply to @neilalexander:dendrite.matrix.org
Layering like this is also a bit slippery. Take for example the difference in how Synapse decodes JSON (decoding into what is effectively a hashmap and tolerating whatever) vs how Dendrite does (decoding into a typed structure and erroring on type mismatches). Unless we're prescribing that JSON has to be decoded in multiple stages for specific event types, which might not be very feasible in some languages, we're not really getting to the root of the issue. The spec has ultimately told us that certain keys with certain names should be certain types, the bottom line is that is the rule everyone should be playing by

TBH I don't think decoding in multiple steps/layers is necessarily the worst idea. We already have to be able to 'decode JSON whilst ignoring unknown keys', you'd conceptually just sort of do that several times.

I agree that enforcing these things harder is the right way to do it, but you're still left with the question of how you apply semantics to mishmashes from before that validation was possible (and this sort of stuff also applies client-side in encrypted payloads I'd say)

09:46:06
@cat:feline.supportCat
In reply to @deepbluev7:neko.dev
And if people turn off that validation, it is fair if servers refuse to join those rooms, because nobody needs to respect tulir's experiments! :p
Doesnt Tulir only turn of CS API level validation?
09:46:15
@tulir:maunium.nettulir
In reply to @cat:feline.support
Doesnt Tulir only turn of CS API level validation?
yes, I follow the mandatory parts of the spec, I only skip the unnecessary checks
09:47:06
@deepbluev7:neko.devNotHSDevNicoRead my previous message and then explain me, where you think that message contradicts the one I sent before it?09:47:49
@cat:feline.supportCatEssentially Tulir is not supposed to break your stuff.09:49:16
@cat:feline.supportCatif we are doing validation properly09:49:22
@cat:feline.supportCatBecause the only validation that matters at the end of the day is the validation that is done at the federation level. 09:49:54
@deepbluev7:neko.devNotHSDevNicoNope, I am free to ignore tulir's events as not useful and possibly malicious. And if tulir messes with the create event, I will just say the room is not joinable09:50:13
@deepbluev7:neko.devNotHSDevNicoI don't have to accept people messing with events for fun (as long as it doesn't cause a room split and not joining a room is not a room split)09:51:00
@deepbluev7:neko.devNotHSDevNicoI spent enough time dealing with sloppy events, it makes my hair go grey and I hate it09:52:12

Show newer messages


Back to Room ListRoom Version: 5