Commit Graph
343 Commits
Author SHA1 Message Date
Athanasius cb60c229a2 tests: Gateway.handle_upload - valid plain message 2022-08-29 16:05:42 +01:00
Athanasius b4deb19e6e tests: Move fixtures to conftest.py, and use in all existing applicable tests 2022-08-29 16:04:18 +01:00
Athanasius bc5f7e9059 Gateway: Factor out the guts of bottle /upload/ route for easier testing 2022-08-29 15:30:27 +01:00
Athanasius 0c42764614 mypy: Moved .mypy.ini into src/ as that's where it'll be used more 2022-08-19 17:28:51 +01:00
Athanasius e15c8e28dd tests/gateway/parse_and_error_handle: Switch to conftest 'plain_journal_scan_valid' 2022-08-19 17:27:48 +01:00
Athanasius 0fed222074 tests/conftest: Fix up whitespace 2022-08-19 17:27:25 +01:00
Athanasius 73cdcc4c60 tests: Move 'plain_no_softwarename' message into conftest.py 2022-08-19 17:21:59 +01:00
Athanasius 4b6f22d9fd tests: Move 'plain_outdated_schema' message into conftest.py 2022-08-19 17:19:56 +01:00
Athanasius 795480f097 tests/Gateway/parse_and_error_handle: Remove stray blank lines 2022-08-19 17:16:01 +01:00
Athanasius eff5cedc2b tests/Gateway/parse_and_error_handle: Working with single eddn.Gateway import
Unfortunately the `eddn_gateway()` fixture needs to be `module` scope, and
that means you can't use it with anything that uses a fixture that utilises
`monkeypatch`, because that's `function` scope.

So, back to directly appending to `sys.path`.
2022-08-19 17:13:10 +01:00
Athanasius 660dad4ff0 tests/conftest: Add 'invalid_json' message 2022-08-19 17:12:35 +01:00
Athanasius 6f17b937e1 tests/Gateway/parse_and_error_handle: Mostly fix flake8/mypy
Now it only complains about the multi-line strings for the test messages.

But those are about to move to the conftest.py dictionary.
2022-08-19 16:39:52 +01:00
Athanasius 9d9b38227f tests/gateway/get_decompressed_message: 'plain_message' test working 2022-08-19 16:29:54 +01:00
Athanasius e28ab45be4 Gateway: Pass headers and body into get_decompressed_message()
This makes it easier to write unit tests for this function, not having to
somehow mock up the bottle request/body stuff.
2022-08-19 16:27:22 +01:00
Athanasius 580160b95b Relay: Add TRACE logging & use in key places
* Define our own TRACE logging level ('deeper' than DEBUG).
* Define `.trace()` method to use it.
* `logger.trace(...)` for duplicate or 'sent to Listeners' messages.

NB: mypy is still unhappy about Logger.trace, despite attempting the same
   workaround asin EDMC.  Hence the `  # type: ignore` on uses.
2022-08-19 16:25:30 +01:00
Athanasius 58c68ec17e tests/Gateway/post_upload: Fix assert
* dc_msg is a webtest object, so need to explicitly check its `body`.
* And that `body` is bytes, not str, so test against `b'OK'`.
2022-08-19 15:53:15 +01:00
Athanasius 00fed9b36a Gateway: Split some of main() into setup_bottle_app()
* This *should* facilitate some easier testing.
2022-08-19 15:13:33 +01:00
Athanasius 8a5bd46c33 tests/config: Gateway settings file
* Currently sets TLS cert/key files to `""` so as to be TLS-free.
2022-08-19 15:08:21 +01:00
Athanasius 6b339e3b11 tests/gateway/post_upload: Use webtest.TestApp for functional tests 2022-08-19 15:07:24 +01:00
Athanasius 30282ba95e tests/conftest: pytest top-level configuration, mostly Fixtures
* `eddn_message()` fixture, set up to return a method when used, such that
  *that* can be called with a key to look up the approproiate test message.
* `test_messages` dictionary to support that.
2022-08-19 15:01:43 +01:00
Athanasius 87c79a3427 tests: Gateway: parse_and_error_handle: Valid journal/scan message 2022-08-18 17:55:27 +01:00
Athanasius ba43199a5d tests: Gateway: parse_and_error_handle: No softwareName 2022-08-18 17:54:50 +01:00
Athanasius ddf9f3524f tests: Gateway: parse_and_error_handle: Two tests to get started
This is Athanasius feeling out how best to add tests to this ancient code.

* Test that eddn.Gateway.parse_and_error_handle() returns the correct error
  string for:
  - Invalid JSON being passed in.
  - An outdated schema being cited.
2022-08-18 17:23:08 +01:00
Athanasius 76251cfd30 Settings: Correct comment typos about reverse proxy setup 2022-08-18 16:20:51 +01:00
Athanasius af9089ccee Tweak Gateway, Monitor, Relay startup logging 2022-08-18 16:19:27 +01:00
Athanasius 5365d5efd6 Bouncer: Remove defunct form-encoded processing
And as a result, no need for `import urlparse`
2022-08-18 16:14:25 +01:00
Athanasius 1f991e7c44 Gateway: Remove un-necessary import urlparse 2022-08-18 16:13:59 +01:00
Athanasius 1384241f13 Fix "are you running this correctly?" check to utilise pathlib
As it was this only worked on systems with `/`-delimited path components.

So use `pathlib.Path.as_posix()` to ensure it *is* in that format.
2022-08-18 15:20:22 +01:00
Athanasius 2310d9cb46 Relay: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius 86cfed0ec5 Monitor: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius 3f024f965c Bouncer: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius 6a4bad49c8 Gateway: Add 'are we running this correctly?' check
Also some misc import order fixups
2022-08-18 15:20:20 +01:00
Athanasius b437df5af8 Move main scripts back into src/eddn/
So, yes, running any of these absolutely directly:

    cd src
    python eddn/Gateway.py

will fail.  But you *can*:

    cd src
    python -m eddn.Gateway

and they work perfectly fine.
2022-08-18 15:20:19 +01:00
Athanasius 7ad46d409b Rename main scripts to have "EDDN_" prefix
This avoids having possibly clashing files in the environment
sites-packages/, i.e. 'Gateway.py' is a bit too generic.
2022-08-18 15:20:18 +01:00
Athanasius 0f04fd61a5 All change yet again: We're now building and installing a wheel, not an egg
# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius c2dc30c8a2 Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius 47ae96ddc1 More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json

# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius d36e45873d Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:17 +01:00
Athanasius 1ae349a4b7 Make per-component logging more obvious 2022-08-18 15:20:15 +01:00
Athanasius 4d705e4f37 Settings: Default Relay and Monitor to localhost listen. 2022-08-18 15:20:15 +01:00
Athanasius 95b56ee3b0 Relay: Allow for running without TLS 2022-08-18 15:20:15 +01:00
Athanasius d79f4a5aa1 Monitor: Change to be able to run TLS-less
NB: Not yet changed the actual monitor web page files.
2022-08-18 15:20:15 +01:00
Athanasius 128dab965c TLS: Optionally use TLS if you set non-empty CERT_FILE and KEY_FILE
Whilst we do want to go TLS-less in the actual EDDN code, for ease of
setting up automated end to end functional testing, leave the
possibility of running with TLS termination as well.
2022-08-18 15:20:15 +01:00
Athanasius 83e2b548ad Gateway: Use a dict for kwargs to app.run()
This is so we can now adjust if we're putting the TLS cert/key files in.
2022-08-18 15:20:14 +01:00
Athanasius d5dbc3262f Gateway: Put remote_addr in [], as it could be multiple, comma-separated
# Conflicts:
#	src/eddn/Gateway.py

# Conflicts:
#	src/eddn/Gateway.py
2022-08-18 15:20:12 +01:00
Athanasius 9bf24f9a05 EDDNWSGIHandler: X-Forwarded-For can be a comma-separated list
So, make it easier to pull out the IPs, single or not.
2022-08-18 15:19:58 +01:00
Athanasius 8cf0d12d97 Move EDDNWSGIHandler into its own file 2022-08-18 15:19:58 +01:00
Athanasius 402758f1d6 Gateway: Fixed logging to be consistent and use client IP
* Send all the bottle server output through our logger.
* Ensure gevent uses client IP, not 127.0.0.1.
2022-08-18 15:19:58 +01:00
Athanasius 4b0898122d Gateway: Don't use TLS cert in app setup
* The /upload/ functionality continues to work.
2022-08-18 15:19:57 +01:00
Athanasius 8b5b9142a3 Settings: Remove {CERT,KEY}_FILE as first step to no more TLS 2022-08-18 15:19:57 +01:00