From 7819eff82665304aed3f6cc966025e66198b8cf0 Mon Sep 17 00:00:00 2001 From: Athanasius Date: Thu, 20 Jan 2022 11:32:08 +0000 Subject: [PATCH 1/3] schemas/README: Give some hints about odyssey/horizons flags --- schemas/README-EDDN-schemas.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/schemas/README-EDDN-schemas.md b/schemas/README-EDDN-schemas.md index b4651d8..bbdbc43 100644 --- a/schemas/README-EDDN-schemas.md +++ b/schemas/README-EDDN-schemas.md @@ -245,6 +245,17 @@ Each `message` object must have, at bare minimum: will be much more than this. Consult the [schemas and their documentation](./). +Where the schema allows for them, add the `horizons` and `odyssey` +keys with boolean values. `null` is not allowed in the values, so if +you cannot determine a value do not include that key at all. The best +source of these is the `LoadGame` event from journals. It's present +both in the PC local files and the CAPI journal data. If you're +composing a shipyard or outfitting message from CAPI data then it is +possible to synthesise the `horizons` flag. For now consult the function +`capi_is_horizons()` in +[EDMarketConnector:plugins/eddn.py](https://github.com/EDCD/EDMarketConnector/blob/stable/plugins/eddn.py) +for a method to achieve this. + Because the first versions of some schemas were defined when only the CAPI data was available, before Journal files existed, many of the key names chosen in the schemas are based on the equivalent in CAPI data, not Journal events. From 853c309df0a104d755acdb7f18194709c36f0a5e Mon Sep 17 00:00:00 2001 From: Athanasius Date: Thu, 20 Jan 2022 11:40:41 +0000 Subject: [PATCH 2/3] schemas/README: Give examples of `LoadGame` for `Horizons`/`Odyssey` flags --- schemas/README-EDDN-schemas.md | 44 +++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/schemas/README-EDDN-schemas.md b/schemas/README-EDDN-schemas.md index bbdbc43..50c1dfd 100644 --- a/schemas/README-EDDN-schemas.md +++ b/schemas/README-EDDN-schemas.md @@ -245,17 +245,6 @@ Each `message` object must have, at bare minimum: will be much more than this. Consult the [schemas and their documentation](./). -Where the schema allows for them, add the `horizons` and `odyssey` -keys with boolean values. `null` is not allowed in the values, so if -you cannot determine a value do not include that key at all. The best -source of these is the `LoadGame` event from journals. It's present -both in the PC local files and the CAPI journal data. If you're -composing a shipyard or outfitting message from CAPI data then it is -possible to synthesise the `horizons` flag. For now consult the function -`capi_is_horizons()` in -[EDMarketConnector:plugins/eddn.py](https://github.com/EDCD/EDMarketConnector/blob/stable/plugins/eddn.py) -for a method to achieve this. - Because the first versions of some schemas were defined when only the CAPI data was available, before Journal files existed, many of the key names chosen in the schemas are based on the equivalent in CAPI data, not Journal events. @@ -279,6 +268,39 @@ changes to your code. It is also advisable to Watch this repository on GitHub so as to be aware of any changes to schemas. +#### `horizons` and `odyssey` flags + +Where the schema allows for them, add the `horizons` and `odyssey` +keys with boolean values. `null` is not allowed in the values, so if +you cannot determine a value do not include that key at all. The best +source of these is the `LoadGame` event from journals. It's present +both in the PC local files and the CAPI journal data. If you're +composing a shipyard or outfitting message from CAPI data then it is +possible to synthesise the `horizons` flag. For now consult the function +`capi_is_horizons()` in +[EDMarketConnector:plugins/eddn.py](https://github.com/EDCD/EDMarketConnector/blob/stable/plugins/eddn.py) +for a method to achieve this. + +As of 2022-01-22 the following was observed for the `LoadGame` events as +present in CAPI-sourced Journal files (which were confirmed to match the +PC-local files for these events): + +- PC Odyssey Client, game version `4.0.0.1002`: + ```json + { "timestamp":"2022-01-20T11:15:22Z", "event":"LoadGame", "FID":"F", "Commander":"", "Horizons":true, "Odyssey":true,... + ``` +- PC Horizons Client, game version `3.8.0.403`, no `Odyssey` key was + present: + ```json + { "timestamp":"2022-01-20T11:20:17Z", "event":"LoadGame", "FID":"F", "Commander":"", "Horizons":true,... + ``` + +- PC 'base' Client, game version `3.8.0.403`, no `Odyssey` key was + present: + ```json + { "timestamp":"2022-01-20T11:22:54Z", "event":"LoadGame", "FID":"F", "Commander":"", "Horizons":false,... + ``` + ### Server responses There are three possible sources of HTTP responses when sending an upload to EDDN. From a8f657c2d6a5354df19df778780cbc2f75601174 Mon Sep 17 00:00:00 2001 From: Athanasius Date: Fri, 21 Jan 2022 09:43:55 +0000 Subject: [PATCH 3/3] schemas/README: bottle body size limit *is* affected by compression I explicitly tested this with a 50KiB limit on beta EDDN service: 1. 49973 byte message does make it through, in plain text. 2. 51312 byte message, no compression, 413. 3. Same message, with gzip compression, 200 4. The 149242 byte file I based those extra ones on also makes it through when compressed. Indeed the Gateway logging cites the compressed-body size. `bottle` isn't looking at the message and magically decompressing it in order to check plain-text size. --- schemas/README-EDDN-schemas.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/schemas/README-EDDN-schemas.md b/schemas/README-EDDN-schemas.md index 50c1dfd..3de98f6 100644 --- a/schemas/README-EDDN-schemas.md +++ b/schemas/README-EDDN-schemas.md @@ -332,8 +332,10 @@ make a valid request" responses you might experience the following: #### bottle responses 1. `413` - `Payload Too Large` - `bottle` enforces a maximum request size - and the request exceeds that. As of 2022-01-07 the limit is 1MiB, and - pertains to the plain-text size, not after gzip compression if used. + and the request exceeds that. As of 2022-01-07 the limit is 1MiB, + which is versus the compressed size of the body, if compression is + used. Thus compression *will* allow for sending approximately 10x + larger messages. To verify the current limit check for the line that looks like: ```