326 Commits

Author SHA1 Message Date
Athanasius
8a5bd46c33
tests/config: Gateway settings file
* Currently sets TLS cert/key files to `""` so as to be TLS-free.
2022-08-19 15:08:21 +01:00
Athanasius
6b339e3b11
tests/gateway/post_upload: Use webtest.TestApp for functional tests 2022-08-19 15:07:24 +01:00
Athanasius
30282ba95e
tests/conftest: pytest top-level configuration, mostly Fixtures
* `eddn_message()` fixture, set up to return a method when used, such that
  *that* can be called with a key to look up the approproiate test message.
* `test_messages` dictionary to support that.
2022-08-19 15:01:43 +01:00
Athanasius
87c79a3427
tests: Gateway: parse_and_error_handle: Valid journal/scan message 2022-08-18 17:55:27 +01:00
Athanasius
ba43199a5d
tests: Gateway: parse_and_error_handle: No softwareName 2022-08-18 17:54:50 +01:00
Athanasius
ddf9f3524f
tests: Gateway: parse_and_error_handle: Two tests to get started
This is Athanasius feeling out how best to add tests to this ancient code.

* Test that eddn.Gateway.parse_and_error_handle() returns the correct error
  string for:
  - Invalid JSON being passed in.
  - An outdated schema being cited.
2022-08-18 17:23:08 +01:00
Athanasius
76251cfd30
Settings: Correct comment typos about reverse proxy setup 2022-08-18 16:20:51 +01:00
Athanasius
af9089ccee
Tweak Gateway, Monitor, Relay startup logging 2022-08-18 16:19:27 +01:00
Athanasius
5365d5efd6
Bouncer: Remove defunct form-encoded processing
And as a result, no need for `import urlparse`
2022-08-18 16:14:25 +01:00
Athanasius
1f991e7c44
Gateway: Remove un-necessary import urlparse 2022-08-18 16:13:59 +01:00
Athanasius
1384241f13
Fix "are you running this correctly?" check to utilise pathlib
As it was this only worked on systems with `/`-delimited path components.

So use `pathlib.Path.as_posix()` to ensure it *is* in that format.
2022-08-18 15:20:22 +01:00
Athanasius
2310d9cb46
Relay: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius
86cfed0ec5
Monitor: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius
3f024f965c
Bouncer: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius
6a4bad49c8
Gateway: Add 'are we running this correctly?' check
Also some misc import order fixups
2022-08-18 15:20:20 +01:00
Athanasius
b437df5af8
Move main scripts back into src/eddn/
So, yes, running any of these absolutely directly:

    cd src
    python eddn/Gateway.py

will fail.  But you *can*:

    cd src
    python -m eddn.Gateway

and they work perfectly fine.
2022-08-18 15:20:19 +01:00
Athanasius
7ad46d409b
Rename main scripts to have "EDDN_" prefix
This avoids having possibly clashing files in the environment
sites-packages/, i.e. 'Gateway.py' is a bit too generic.
2022-08-18 15:20:18 +01:00
Athanasius
0f04fd61a5
All change yet again: We're now building and installing a wheel, not an egg
# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius
c2dc30c8a2
Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius
47ae96ddc1
More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json

# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius
d36e45873d
Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:17 +01:00
Athanasius
1ae349a4b7
Make per-component logging more obvious 2022-08-18 15:20:15 +01:00
Athanasius
4d705e4f37
Settings: Default Relay and Monitor to localhost listen. 2022-08-18 15:20:15 +01:00
Athanasius
95b56ee3b0
Relay: Allow for running without TLS 2022-08-18 15:20:15 +01:00
Athanasius
d79f4a5aa1
Monitor: Change to be able to run TLS-less
NB: Not yet changed the actual monitor web page files.
2022-08-18 15:20:15 +01:00
Athanasius
128dab965c
TLS: Optionally use TLS if you set non-empty CERT_FILE and KEY_FILE
Whilst we do want to go TLS-less in the actual EDDN code, for ease of
setting up automated end to end functional testing, leave the
possibility of running with TLS termination as well.
2022-08-18 15:20:15 +01:00
Athanasius
83e2b548ad
Gateway: Use a dict for kwargs to app.run()
This is so we can now adjust if we're putting the TLS cert/key files in.
2022-08-18 15:20:14 +01:00
Athanasius
d5dbc3262f
Gateway: Put remote_addr in [], as it could be multiple, comma-separated
# Conflicts:
#	src/eddn/Gateway.py

# Conflicts:
#	src/eddn/Gateway.py
2022-08-18 15:20:12 +01:00
Athanasius
9bf24f9a05
EDDNWSGIHandler: X-Forwarded-For can be a comma-separated list
So, make it easier to pull out the IPs, single or not.
2022-08-18 15:19:58 +01:00
Athanasius
8cf0d12d97
Move EDDNWSGIHandler into its own file 2022-08-18 15:19:58 +01:00
Athanasius
402758f1d6
Gateway: Fixed logging to be consistent and use client IP
* Send all the bottle server output through our logger.
* Ensure gevent uses client IP, not 127.0.0.1.
2022-08-18 15:19:58 +01:00
Athanasius
4b0898122d
Gateway: Don't use TLS cert in app setup
* The /upload/ functionality continues to work.
2022-08-18 15:19:57 +01:00
Athanasius
8b5b9142a3
Settings: Remove {CERT,KEY}_FILE as first step to no more TLS 2022-08-18 15:19:57 +01:00
Athanasius
71ab6ba6be
Gateway: Remove extraneous noqa's 2022-08-18 15:19:56 +01:00
Athanasius
bc5b7db0b1
core/StatsCollector.py: black pass 2022-08-18 15:19:55 +01:00
Athanasius
8a07ae1cdc
core/Validator: black pass, and remove extraneous , in constants 2022-08-18 15:19:55 +01:00
Athanasius
50c87ecf21
core/DuplicateMessages: black / quotes pass 2022-08-18 15:19:55 +01:00
Athanasius
d87344459a
conf/Settings: s/'/"/g; But not with black
We **do** want the special layout of the actual config defaults, so
screw black!
2022-08-18 15:19:54 +01:00
Athanasius
7a6314cba8
conf/__init__.py: docstring 2022-08-18 15:19:54 +01:00
Athanasius
d87106fe26
Relay: Re-format with black 2022-08-18 15:19:54 +01:00
Athanasius
83fa055722
Monitor: Re-format with black 2022-08-18 15:19:54 +01:00
Athanasius
03525efe82
Bouncer: Re-format with black. 2022-08-18 15:19:53 +01:00
Athanasius
5d3358417d
Gateway: Format with black
# Conflicts:
#	src/eddn/Gateway.py
2022-08-18 15:19:50 +01:00
Athanasius
841c7e979e
Gateway: Use "" for strings throughout
# Conflicts:
#	src/eddn/Gateway.py

# Conflicts:
#	src/eddn/Gateway.py
2022-08-18 15:19:33 +01:00
Athanasius
710223494d
src/eddn/core/__init__.py: docstring 2022-08-18 15:19:17 +01:00
Athanasius
47a428c5d1
Bouncer: Use "" for strings 2022-08-18 15:19:17 +01:00
Athanasius
d540078b66
conf/Version: Bump to 3.0 2022-08-18 15:19:17 +01:00
Athanasius
4e6a6fc869
conf/Settings: flake8 and mypy pass 2022-08-18 15:19:17 +01:00
Athanasius
6e7f69ab95
Relay: flake8 and mypy pass 2022-08-18 15:19:16 +01:00
Athanasius
35fcbd89d9
Monitor: flake8 and mypy pass 2022-08-18 15:19:16 +01:00