1082 Commits

Author SHA1 Message Date
Athanasius
e15c8e28dd
tests/gateway/parse_and_error_handle: Switch to conftest 'plain_journal_scan_valid' 2022-08-19 17:27:48 +01:00
Athanasius
0fed222074
tests/conftest: Fix up whitespace 2022-08-19 17:27:25 +01:00
Athanasius
73cdcc4c60
tests: Move 'plain_no_softwarename' message into conftest.py 2022-08-19 17:21:59 +01:00
Athanasius
4b6f22d9fd
tests: Move 'plain_outdated_schema' message into conftest.py 2022-08-19 17:19:56 +01:00
Athanasius
795480f097
tests/Gateway/parse_and_error_handle: Remove stray blank lines 2022-08-19 17:16:01 +01:00
Athanasius
eff5cedc2b
tests/Gateway/parse_and_error_handle: Working with single eddn.Gateway import
Unfortunately the `eddn_gateway()` fixture needs to be `module` scope, and
that means you can't use it with anything that uses a fixture that utilises
`monkeypatch`, because that's `function` scope.

So, back to directly appending to `sys.path`.
2022-08-19 17:13:10 +01:00
Athanasius
660dad4ff0
tests/conftest: Add 'invalid_json' message 2022-08-19 17:12:35 +01:00
Athanasius
6f17b937e1
tests/Gateway/parse_and_error_handle: Mostly fix flake8/mypy
Now it only complains about the multi-line strings for the test messages.

But those are about to move to the conftest.py dictionary.
2022-08-19 16:39:52 +01:00
Athanasius
9d9b38227f
tests/gateway/get_decompressed_message: 'plain_message' test working 2022-08-19 16:29:54 +01:00
Athanasius
e28ab45be4
Gateway: Pass headers and body into get_decompressed_message()
This makes it easier to write unit tests for this function, not having to
somehow mock up the bottle request/body stuff.
2022-08-19 16:27:22 +01:00
Athanasius
580160b95b
Relay: Add TRACE logging & use in key places
* Define our own TRACE logging level ('deeper' than DEBUG).
* Define `.trace()` method to use it.
* `logger.trace(...)` for duplicate or 'sent to Listeners' messages.

NB: mypy is still unhappy about Logger.trace, despite attempting the same
   workaround asin EDMC.  Hence the `  # type: ignore` on uses.
2022-08-19 16:25:30 +01:00
Athanasius
58c68ec17e
tests/Gateway/post_upload: Fix assert
* dc_msg is a webtest object, so need to explicitly check its `body`.
* And that `body` is bytes, not str, so test against `b'OK'`.
2022-08-19 15:53:15 +01:00
Athanasius
00fed9b36a
Gateway: Split some of main() into setup_bottle_app()
* This *should* facilitate some easier testing.
2022-08-19 15:13:33 +01:00
Athanasius
8a5bd46c33
tests/config: Gateway settings file
* Currently sets TLS cert/key files to `""` so as to be TLS-free.
2022-08-19 15:08:21 +01:00
Athanasius
6b339e3b11
tests/gateway/post_upload: Use webtest.TestApp for functional tests 2022-08-19 15:07:24 +01:00
Athanasius
30282ba95e
tests/conftest: pytest top-level configuration, mostly Fixtures
* `eddn_message()` fixture, set up to return a method when used, such that
  *that* can be called with a key to look up the approproiate test message.
* `test_messages` dictionary to support that.
2022-08-19 15:01:43 +01:00
Athanasius
87c79a3427
tests: Gateway: parse_and_error_handle: Valid journal/scan message 2022-08-18 17:55:27 +01:00
Athanasius
ba43199a5d
tests: Gateway: parse_and_error_handle: No softwareName 2022-08-18 17:54:50 +01:00
Athanasius
ddf9f3524f
tests: Gateway: parse_and_error_handle: Two tests to get started
This is Athanasius feeling out how best to add tests to this ancient code.

* Test that eddn.Gateway.parse_and_error_handle() returns the correct error
  string for:
  - Invalid JSON being passed in.
  - An outdated schema being cited.
2022-08-18 17:23:08 +01:00
Athanasius
76251cfd30
Settings: Correct comment typos about reverse proxy setup 2022-08-18 16:20:51 +01:00
Athanasius
af9089ccee
Tweak Gateway, Monitor, Relay startup logging 2022-08-18 16:19:27 +01:00
Athanasius
5365d5efd6
Bouncer: Remove defunct form-encoded processing
And as a result, no need for `import urlparse`
2022-08-18 16:14:25 +01:00
Athanasius
1f991e7c44
Gateway: Remove un-necessary import urlparse 2022-08-18 16:13:59 +01:00
Athanasius
1384241f13
Fix "are you running this correctly?" check to utilise pathlib
As it was this only worked on systems with `/`-delimited path components.

So use `pathlib.Path.as_posix()` to ensure it *is* in that format.
2022-08-18 15:20:22 +01:00
Athanasius
98603273cc
requirements.txt: Add strict-rfc3339==0.7 2022-08-18 15:20:21 +01:00
Athanasius
2310d9cb46
Relay: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius
86cfed0ec5
Monitor: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius
3f024f965c
Bouncer: Add 'are you running this correctly?' check 2022-08-18 15:20:21 +01:00
Athanasius
6a4bad49c8
Gateway: Add 'are we running this correctly?' check
Also some misc import order fixups
2022-08-18 15:20:20 +01:00
Athanasius
fb53653430
pre-commit: Specify more flake8 dependencies 2022-08-18 15:20:20 +01:00
Athanasius
1afa0b9c44
pre-commit: Specify flake8-isort dependency 2022-08-18 15:20:20 +01:00
Athanasius
b9e97a15df
pre-commit: Change flake8 to repo-based 2022-08-18 15:20:20 +01:00
Athanasius
84a19811aa
pre-commit: Additional mypy dependencies & update versions 2022-08-18 15:20:20 +01:00
Athanasius
cba052de38
requirements: Sanitised with 'pip-chill' and bumped to 'eager' versions 2022-08-18 15:20:19 +01:00
Athanasius
1e4cb3b1ab
Add helper scripts for running from source, and document them. 2022-08-18 15:20:19 +01:00
Athanasius
b437df5af8
Move main scripts back into src/eddn/
So, yes, running any of these absolutely directly:

    cd src
    python eddn/Gateway.py

will fail.  But you *can*:

    cd src
    python -m eddn.Gateway

and they work perfectly fine.
2022-08-18 15:20:19 +01:00
Athanasius
242fdff00c
docs: General update for switching to wheel for install 2022-08-18 15:20:19 +01:00
Athanasius
a55d1c3616
docs/Running: Assume python3 for virtualenv 2022-08-18 15:20:19 +01:00
Athanasius
7ad46d409b
Rename main scripts to have "EDDN_" prefix
This avoids having possibly clashing files in the environment
sites-packages/, i.e. 'Gateway.py' is a bit too generic.
2022-08-18 15:20:18 +01:00
Athanasius
0f04fd61a5
All change yet again: We're now building and installing a wheel, not an egg
# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius
ef2c184710
setup.py: Remove comment about pinning module version, as we're not
And we're on python3 now anyway.
2022-08-18 15:20:18 +01:00
Athanasius
c2dc30c8a2
Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius
47ae96ddc1
More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json

# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius
d36e45873d
Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:17 +01:00
Athanasius
ede9abb95e
contrib: apache-eddn.conf: Attempt to correct for new "no direct access"
As we've changed things to default to *not* doing TLS on the actual
services, and thus relying on reverse proxying of all the relevant
end points, this needed updating.

Not yet tested, that will come when I put this on dev.eddn.edcd.io.
2022-08-18 15:20:17 +01:00
Athanasius
8ca813c6ab
monitor: Make example eddn-config.js fit the usual live setup 2022-08-18 15:20:17 +01:00
Athanasius
0473e18633
monitor/schemas.html: Actually, those bits are schema URLs, so revert change
It's only the monitor endpoint URL that needed to be configurable.
Anyone using this wholesale for schemas not under eddn.edcd.io gets to
make the necessary hard-coded changes.
2022-08-18 15:20:17 +01:00
Athanasius
f96e7fc293
monitor/schemas.html: Now supports eddn-config.js 2022-08-18 15:20:16 +01:00
Athanasius
b81931fd3c
setup.py: Backup/restore monitor eddn-config.js if present 2022-08-18 15:20:16 +01:00
Athanasius
db698d70cf
docs/Running: Update for monitor eddn-config.json
Although now I realise the schemas.html is going to need some work.
2022-08-18 15:20:16 +01:00