1242 Commits
Author SHA1 Message Date
Athanasius 6a4bad49c8 Gateway: Add 'are we running this correctly?' check
Also some misc import order fixups
2022-08-18 15:20:20 +01:00
Athanasius fb53653430 pre-commit: Specify more flake8 dependencies 2022-08-18 15:20:20 +01:00
Athanasius 1afa0b9c44 pre-commit: Specify flake8-isort dependency 2022-08-18 15:20:20 +01:00
Athanasius b9e97a15df pre-commit: Change flake8 to repo-based 2022-08-18 15:20:20 +01:00
Athanasius 84a19811aa pre-commit: Additional mypy dependencies & update versions 2022-08-18 15:20:20 +01:00
Athanasius cba052de38 requirements: Sanitised with 'pip-chill' and bumped to 'eager' versions 2022-08-18 15:20:19 +01:00
Athanasius 1e4cb3b1ab Add helper scripts for running from source, and document them. 2022-08-18 15:20:19 +01:00
Athanasius b437df5af8 Move main scripts back into src/eddn/
So, yes, running any of these absolutely directly:

    cd src
    python eddn/Gateway.py

will fail.  But you *can*:

    cd src
    python -m eddn.Gateway

and they work perfectly fine.
2022-08-18 15:20:19 +01:00
Athanasius 242fdff00c docs: General update for switching to wheel for install 2022-08-18 15:20:19 +01:00
Athanasius a55d1c3616 docs/Running: Assume python3 for virtualenv 2022-08-18 15:20:19 +01:00
Athanasius 7ad46d409b Rename main scripts to have "EDDN_" prefix
This avoids having possibly clashing files in the environment
sites-packages/, i.e. 'Gateway.py' is a bit too generic.
2022-08-18 15:20:18 +01:00
Athanasius 0f04fd61a5 All change yet again: We're now building and installing a wheel, not an egg
# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius ef2c184710 setup.py: Remove comment about pinning module version, as we're not
And we're on python3 now anyway.
2022-08-18 15:20:18 +01:00
Athanasius c2dc30c8a2 Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius 47ae96ddc1 More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json

# Conflicts:
#	src/eddn/schemas/fssbodysignals-README.md
#	src/eddn/schemas/fssbodysignals-v1.0.json
#	src/eddn/schemas/fsssignaldiscovered-README.md
#	src/eddn/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:18 +01:00
Athanasius d36e45873d Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.

# Conflicts:
#	src/schemas/fssbodysignals-README.md
#	src/schemas/fssbodysignals-v1.0.json
#	src/schemas/fsssignaldiscovered-README.md
#	src/schemas/fsssignaldiscovered-v1.0.json
2022-08-18 15:20:17 +01:00
Athanasius ede9abb95e contrib: apache-eddn.conf: Attempt to correct for new "no direct access"
As we've changed things to default to *not* doing TLS on the actual
services, and thus relying on reverse proxying of all the relevant
end points, this needed updating.

Not yet tested, that will come when I put this on dev.eddn.edcd.io.
2022-08-18 15:20:17 +01:00
Athanasius 8ca813c6ab monitor: Make example eddn-config.js fit the usual live setup 2022-08-18 15:20:17 +01:00
Athanasius 0473e18633 monitor/schemas.html: Actually, those bits are schema URLs, so revert change
It's only the monitor endpoint URL that needed to be configurable.
Anyone using this wholesale for schemas not under eddn.edcd.io gets to
make the necessary hard-coded changes.
2022-08-18 15:20:17 +01:00
Athanasius f96e7fc293 monitor/schemas.html: Now supports eddn-config.js 2022-08-18 15:20:16 +01:00
Athanasius b81931fd3c setup.py: Backup/restore monitor eddn-config.js if present 2022-08-18 15:20:16 +01:00
Athanasius db698d70cf docs/Running: Update for monitor eddn-config.json
Although now I realise the schemas.html is going to need some work.
2022-08-18 15:20:16 +01:00
Athanasius e01b81b3b9 monitor: Move eddn-config.js to only an example, and make it such
We won't want the live config copied in/overwritten on each invocation
of 'python setup.py install'.
2022-08-18 15:20:16 +01:00
Athanasius 8f73060c74 monitor: Move configuration into separate file
This will allow for running more than one monitor without editing actual
source.
2022-08-18 15:20:16 +01:00
Athanasius 1ae349a4b7 Make per-component logging more obvious 2022-08-18 15:20:15 +01:00
Athanasius 4d705e4f37 Settings: Default Relay and Monitor to localhost listen. 2022-08-18 15:20:15 +01:00
Athanasius 95b56ee3b0 Relay: Allow for running without TLS 2022-08-18 15:20:15 +01:00
Athanasius d79f4a5aa1 Monitor: Change to be able to run TLS-less
NB: Not yet changed the actual monitor web page files.
2022-08-18 15:20:15 +01:00
Athanasius 128dab965c TLS: Optionally use TLS if you set non-empty CERT_FILE and KEY_FILE
Whilst we do want to go TLS-less in the actual EDDN code, for ease of
setting up automated end to end functional testing, leave the
possibility of running with TLS termination as well.
2022-08-18 15:20:15 +01:00
Athanasius 83e2b548ad Gateway: Use a dict for kwargs to app.run()
This is so we can now adjust if we're putting the TLS cert/key files in.
2022-08-18 15:20:14 +01:00
Athanasius d5dbc3262f Gateway: Put remote_addr in [], as it could be multiple, comma-separated
# Conflicts:
#	src/eddn/Gateway.py

# Conflicts:
#	src/eddn/Gateway.py
2022-08-18 15:20:12 +01:00
Athanasius 9bf24f9a05 EDDNWSGIHandler: X-Forwarded-For can be a comma-separated list
So, make it easier to pull out the IPs, single or not.
2022-08-18 15:19:58 +01:00
Athanasius 8cf0d12d97 Move EDDNWSGIHandler into its own file 2022-08-18 15:19:58 +01:00
Athanasius 402758f1d6 Gateway: Fixed logging to be consistent and use client IP
* Send all the bottle server output through our logger.
* Ensure gevent uses client IP, not 127.0.0.1.
2022-08-18 15:19:58 +01:00
Athanasius e613767afa Update docs and apache contrib file for 'no more TLS' 2022-08-18 15:19:57 +01:00
Athanasius 4b0898122d Gateway: Don't use TLS cert in app setup
* The /upload/ functionality continues to work.
2022-08-18 15:19:57 +01:00
Athanasius 8b5b9142a3 Settings: Remove {CERT,KEY}_FILE as first step to no more TLS 2022-08-18 15:19:57 +01:00
Athanasius 56b539fd56 scripts/check-schemas-load: Standardise per-schema output
If we're going to say a schema is OK, then prefix any output with the
schema file location.
2022-08-18 15:19:57 +01:00
Athanasius 93fb5b9345 scripts/check-schemas-load: black-formatted, and be verbose 2022-08-18 15:19:57 +01:00
Athanasius ff18edff6e setup.py: Turn isort off/on around import setup_env.py
setup_env.py is NEVER checked into git, forcing anyone running the code
to ensure they have it set up correctly before setup.py will work.
2022-08-18 15:19:56 +01:00
Athanasius 015b4d1b2c .flake8: Some tuning for EDDN specifically 2022-08-18 15:19:56 +01:00
Athanasius 71ab6ba6be Gateway: Remove extraneous noqa's 2022-08-18 15:19:56 +01:00
Athanasius a1a69dec1b .python-version: Specify only 3.9 in general, not specific patch
We'll be using Debian bullseye's 3.9, which is currently 3.9.2
2022-08-18 15:19:56 +01:00
Athanasius a0ddf4a7b9 requirements-dev: Add "black" 2022-08-18 15:19:55 +01:00
Athanasius bc5b7db0b1 core/StatsCollector.py: black pass 2022-08-18 15:19:55 +01:00
Athanasius 8a07ae1cdc core/Validator: black pass, and remove extraneous , in constants 2022-08-18 15:19:55 +01:00
Athanasius 50c87ecf21 core/DuplicateMessages: black / quotes pass 2022-08-18 15:19:55 +01:00
Athanasius 5e8c17222d setup.py: black and otherwise s/'/"/g pass 2022-08-18 15:19:55 +01:00
Athanasius d823056e15 pyproject.toml: Exclude everything but actual core EDDN source 2022-08-18 15:19:54 +01:00
Athanasius d87344459a conf/Settings: s/'/"/g; But not with black
We **do** want the special layout of the actual config defaults, so
screw black!
2022-08-18 15:19:54 +01:00