718 Commits

Author SHA1 Message Date
Athanasius
8d5b03915b Gateway: Log where upload data is form-encoded
We suspect no-one is actually using this possible form of upload, and
ideally would remove the code for it entirely.  First we need to log any
uploaders that do use this form to be sure.
2022-01-18 15:53:06 +00:00
Athanasius
4ae424f887
Merge pull request #169 from EDCD/enhancement/CL-arg-loglevel
Support `--loglevel <level>` CL arg in all entry point scripts
2022-01-18 15:38:24 +00:00
Athanasius
8f7910a5d9 Bouncer: Support --loglevel <level> CL arg
This also aligns the general logger setup with the other scripts
2022-01-18 15:22:02 +00:00
Athanasius
6986af7ca0 Settings.py: Remove import argparse
CL arg parsing is done in main scripts now.
2022-01-18 15:17:45 +00:00
Athanasius
fe214583aa Relay: Support --loglevel <level> fully
There's no real logging in Relay yet, but this makes it support
switching level anyway.
2022-01-18 15:16:32 +00:00
Athanasius
86b283f920 Monitor.py: Minimal changes to allow --loglevel <level>
The CL arg won't actually *do* anything yet, because this script doesn't
actually have a logger set up at all.  But we need to fake this else
eddn.conf.Settings.loadConfig() won't be called correctly or work.
2022-01-18 15:09:56 +00:00
Athanasius
fb3c0348ad Gateway: Support --loglevel <level> CL arg
NB: This required changes to src/eddn/conf/Settings.py as well, which
means other scripts, e.g. Monitor.py, will also need changing.
2022-01-18 15:02:38 +00:00
Athanasius
872c65c280 setup.py: Do *not* install eddn-logs-archive script
This should be done manually, so as to be using the correct version from
the correct branch on the live service.
2022-01-18 14:16:45 +00:00
Athanasius
79b78ac831 scripts/eddn-report: Quieten some EDMC errors
They're noted in <https://github.com/EDCD/EDMarketConnector/issues/1403>
2022-01-18 11:21:21 +00:00
Athanasius
3ffff54c4b contrib/eddn-logs-archive: Run gateway error report script on gateway.log
* Renamed some variables to be truer to their content and use.
* Run eddn-report-log-errors if rotating a gateway.log file.
2022-01-17 09:22:37 +00:00
Athanasius
a1e330be2c scripts/eddn-report: EVA is abandoned, so ignore 2022-01-17 09:18:40 +00:00
Athanasius
cc5ae32227 scripts/eddn-report: Correct EliteLogAgent comment 2022-01-12 15:43:59 +00:00
Athanasius
e93c04bb21 scripts/eddn-report: EliteLogAgent latest is 2.0.0.660
This allows for ignoring the ancient 0.9.2.412 that showed some errors
yesterday.
2022-01-12 09:40:47 +00:00
Athanasius
00071ba7e9 scripts/testing: Remove per-scenario scripts
The required functionality is now all in `test-sender.py`.
2022-01-11 16:43:30 +00:00
Athanasius
ad8fc57df3 scripts/test-sender: Support --gzip=bad 2022-01-11 16:41:25 +00:00
Athanasius
7537a300c0 scripts/test-sender: Support for valid gzip compression 2022-01-11 16:31:05 +00:00
Athanasius
fb83aae240 scripts/test-sender: Support for requesting good or bad form encoding 2022-01-11 16:27:20 +00:00
Athanasius
ecd3afbbbc scripts/test-sender.py: Allow 'beta' and 'dev' --url aliases
Also explicitly states the URL and input file it uses.
2022-01-11 16:12:21 +00:00
Athanasius
02b2ea6e18 scripts/test-sender.py: Allow for changing target URL 2022-01-11 16:08:59 +00:00
Athanasius
a4eb7548d9
Merge pull request #166 from EDCD/fix/165/bad-formencoded-detection
Gateway: Revert non-gzip form encoded check
2022-01-11 15:39:01 +00:00
Athanasius
6799a7fae9 scripts/testing: Target dev:4432, not beta:4431
In the long-term these all need to support a CL arg to select the
target.
2022-01-11 15:36:53 +00:00
Athanasius
a1d347aec5 scripts/testing: Add general "uses gzip" script 2022-01-11 15:35:24 +00:00
Athanasius
abcf472197 Gateway: Revert non-gzip form encoded check
Fixes the regression identified in #165

However the gzip code path can still erroneously think a decompressed
request body is form encoded when it is not.  This happens when any text
in the decompressed body matches the regex:

        .+=[^\&;]*

i.e. some text, followed by `=`, and then some more text, possibly
empty, followed by an ampersand `&`, or semi-colon `;`, or the end of the
string.

`&` and `;` are used to separate key=value pairs in form encoding, the
`=` separates a key from its value.
2022-01-11 15:06:02 +00:00
Athanasius
6add1d5c6b Merge branch 'master' of github.com:EDCD/EDDN 2022-01-11 15:01:20 +00:00
Athanasius
759b14c844 scripts/testing: Add "valid JSON looks like form-encoded" file
Ref: https://github.com/EDCD/EDDN/issues/165
2022-01-11 14:40:16 +00:00
Athanasius
2d60092d0e
README: Emphasise developers must comply with documentation 2022-01-11 13:47:41 +00:00
Athanasius
1caf5f0600
README: 'Documentation' section removed as un-necessary 2022-01-11 13:44:42 +00:00
Athanasius
01ce718ab8
Merge branch 'master' of github.com:EDSM-NET/EDDN 2022-01-11 13:42:48 +00:00
Athanasius
40262322ff
README: Expansion and tweaks for replacing the Wiki page
* A few uploading PC softwares are listed to lower the barrier for a new
  player finding such.  This is *never* going to be an exhaustive list.
* Added section listing console options for contributing data.  Again,
  not intended to be exhaustive.
* Layout/sections tweaked.  Hopefully the use of double horizontal rules
  emphasises which section text is a part of.
2022-01-11 13:40:42 +00:00
Athanasius
a39f24c55d
README: Add 'Contact' section at the bottom 2022-01-11 13:13:06 +00:00
Athanasius
deb7383e32 docs/schemas: Additions and tweaks for uploaders
* Document the "you tried to use plain HTTP" response from Reverse
Proxy in the appropriate section.

* Switch URL for Gateway.py to `live` branch.

* 'disallowed' Schema Validation error cites **value**, not key *name*.

* Call out the possible need to explicitly subscribe to the empty topic
in order to receive messages... and that currently server-side topic
filtering isn't possible.
2022-01-11 12:48:40 +00:00
Athanasius
1943003b06
README: Complete re-write
* Expand the initial blurb into a full 'About' section.
* Making 'Using EDDN' an actual section, to facilitate migrating away
  from the wiki page.
* Guide players towards some useful information (guide and sites).
* Ensure developers know to check the *live* documentation.
* Explicitly link to the current wiki page and EDDN status.
* Add 'Hosting of the live service' section, and only list the current
  situation (Vivio hosting is 4-5 years ago now, and Anthor's was never
  called out).
2022-01-11 12:24:33 +00:00
Athanasius
084ea35286 setup.py: Change "live branch for live" check to use setup_env.py
The whole build/install process is dependent on the EDDN_ENV value
anyway, so we might as well use it.

This way if a future setup doesn't have a specific string in CWD this
check doesn't break.

Yes, this whole check means you MUST be building from a git checkout,
not the files sourced by some other means.
2022-01-10 16:23:14 +00:00
Athanasius
35c4f89c68 setup.py: Detect if not using live branch in live directory 2022-01-10 16:17:39 +00:00
Athanasius
495e27861e scripts: Make it obvious that test-bad-gzip.py doesn't work 2022-01-10 15:53:44 +00:00
Athanasius
a8098f45b9 Move eddn-report-log-errors into scripts/ 2022-01-10 09:42:17 +00:00
Athanasius
aa7fabf134 Merge branch 'master' into develop 2022-01-10 09:40:14 +00:00
Athanasius
66d39a1bba Merge branch 'master' into develop 2022-01-10 09:38:53 +00:00
Athanasius
f31974d6ff
Merge pull request #162 from EDCD/enhancement/161/gateway-improve-misc-error-reports
Gateway: Properly report all errors to uploaders
2022-01-09 16:41:15 +00:00
Athanasius
ddb9480ee6
Gateway/testscripts: Add a small README 2022-01-09 16:27:08 +00:00
Athanasius
2a8eb8d021
Gateway: Add test scripts and supporting files
These are what I was using on my home server to test the prior code
changes to how the Gateway code reports errors.

Ultimately these should become a part of proper tests, but for now
they're at least in the repository for anyone to utilise.
2022-01-09 16:21:23 +00:00
Athanasius
75c117fb12
docs/schemas: form-encoded messages are deprecated and not documented
At some point I might add logging to see if any sender even *is*
utilising this, and if not actually remove the possibility entirely.
2022-01-09 16:14:00 +00:00
Athanasius
11d3501c59
Gateway/errors: More documentation tweaking 2022-01-09 16:13:01 +00:00
Athanasius
4ad1519d1d
Gateway/errors: Improve general outline of documentation section 2022-01-09 16:10:01 +00:00
Athanasius
ccde820ba7
Gateway/errors: 'Schema Validation' properly tagged and documented 2022-01-09 16:02:28 +00:00
Athanasius
e111fb8415
Gateway/errors: Outdate Schema: Tweak message and document 2022-01-09 15:48:27 +00:00
Athanasius
e931bfff96
Gateway/errors: MalformedUploadError isn't raised upon JSON issue 2022-01-09 15:45:56 +00:00
Athanasius
b9b59329d5
Gateway/errors: Document 'Malformed Upload' error 2022-01-09 15:43:51 +00:00
Athanasius
4da60215f0
Gateway/errors: Tag FAIL if a zlib.error 2022-01-09 15:43:12 +00:00
Athanasius
8edae919e2
Gateway/errors: Call out if FAIL is specifically on JSON parsing 2022-01-09 15:38:29 +00:00