1242 Commits
Author SHA1 Message Date
Athanasius 56ab2d9445 pre-commit: Change flake8 to repo-based 2022-03-18 14:18:09 +00:00
Athanasius 07ab15102e pre-commit: Additional mypy dependencies & update versions 2022-03-18 14:04:24 +00:00
Athanasius ce69f2ef4b requirements: Sanitised with 'pip-chill' and bumped to 'eager' versions 2022-03-18 12:19:52 +00:00
Athanasius 5b110fa554 Add helper scripts for running from source, and document them. 2022-03-18 11:56:56 +00:00
Athanasius f8872c6dfa Move main scripts back into src/eddn/
So, yes, running any of these absolutely directly:

    cd src
    python eddn/Gateway.py

will fail.  But you *can*:

    cd src
    python -m eddn.Gateway

and they work perfectly fine.
2022-03-18 11:41:51 +00:00
Athanasius 5f64fb1c33 docs: General update for switching to wheel for install 2022-03-14 17:03:30 +00:00
Athanasius c6e5820683 docs/Running: Assume python3 for virtualenv 2022-03-14 16:53:18 +00:00
Athanasius f5879fbce7 Rename main scripts to have "EDDN_" prefix
This avoids having possibly clashing files in the environment
sites-packages/, i.e. 'Gateway.py' is a bit too generic.
2022-03-14 16:17:22 +00:00
Athanasius 8345675b89 All change yet again: We're now building and installing a wheel, not an egg 2022-03-14 16:05:09 +00:00
Athanasius cb5766a0eb setup.py: Remove comment about pinning module version, as we're not
And we're on python3 now anyway.
2022-03-14 14:55:16 +00:00
Athanasius 9a1cc7b141 Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.
2022-03-14 14:51:11 +00:00
Athanasius 39058653cb More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json
2022-03-14 14:21:22 +00:00
Athanasius c402617760 Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.
2022-03-14 13:12:58 +00:00
Athanasius 0e8e22a146 contrib: apache-eddn.conf: Attempt to correct for new "no direct access"
As we've changed things to default to *not* doing TLS on the actual
services, and thus relying on reverse proxying of all the relevant
end points, this needed updating.

Not yet tested, that will come when I put this on dev.eddn.edcd.io.
2022-03-13 16:56:21 +00:00
Athanasius bd3fdce53b monitor: Make example eddn-config.js fit the usual live setup 2022-03-13 16:46:11 +00:00
Athanasius 0916d0c440 monitor/schemas.html: Actually, those bits are schema URLs, so revert change
It's only the monitor endpoint URL that needed to be configurable.
Anyone using this wholesale for schemas not under eddn.edcd.io gets to
make the necessary hard-coded changes.
2022-03-13 16:41:04 +00:00
Athanasius 11ac577e34 monitor/schemas.html: Now supports eddn-config.js 2022-03-13 16:37:30 +00:00
Athanasius 6d4d449bb7 setup.py: Backup/restore monitor eddn-config.js if present 2022-03-13 16:33:56 +00:00
Athanasius d953153d0b docs/Running: Update for monitor eddn-config.json
Although now I realise the schemas.html is going to need some work.
2022-03-13 16:07:08 +00:00
Athanasius 6e88a638a5 monitor: Move eddn-config.js to only an example, and make it such
We won't want the live config copied in/overwritten on each invocation
of 'python setup.py install'.
2022-03-13 15:57:41 +00:00
Athanasius 063322df4f monitor: Move configuration into separate file
This will allow for running more than one monitor without editing actual
source.
2022-03-13 15:55:03 +00:00
Athanasius 8c62afb869 Make per-component logging more obvious 2022-03-13 15:42:53 +00:00
Athanasius 52497dfc38 Settings: Default Relay and Monitor to localhost listen. 2022-03-13 15:34:54 +00:00
Athanasius d0a98fe1b7 Relay: Allow for running without TLS 2022-03-13 15:34:09 +00:00
Athanasius e8fa824b9d Monitor: Change to be able to run TLS-less
NB: Not yet changed the actual monitor web page files.
2022-03-13 13:59:15 +00:00
Athanasius 1166b814b5 TLS: Optionally use TLS if you set non-empty CERT_FILE and KEY_FILE
Whilst we do want to go TLS-less in the actual EDDN code, for ease of
setting up automated end to end functional testing, leave the
possibility of running with TLS termination as well.
2022-03-13 13:46:37 +00:00
Athanasius 80cae12a4c Gateway: Use a dict for kwargs to app.run()
This is so we can now adjust if we're putting the TLS cert/key files in.
2022-03-13 13:32:13 +00:00
Athanasius 82ad3d54c7 Gateway: Put remote_addr in [], as it could be multiple, comma-separated 2022-03-12 19:02:34 +00:00
Athanasius b137d8c234 EDDNWSGIHandler: X-Forwarded-For can be a comma-separated list
So, make it easier to pull out the IPs, single or not.
2022-03-12 19:00:07 +00:00
Athanasius c415e7c52a Move EDDNWSGIHandler into its own file 2022-03-12 17:12:17 +00:00
Athanasius 4d2512750a Gateway: Fixed logging to be consistent and use client IP
* Send all the bottle server output through our logger.
* Ensure gevent uses client IP, not 127.0.0.1.
2022-03-12 17:06:37 +00:00
Athanasius d7e4425b1d Update docs and apache contrib file for 'no more TLS' 2022-03-12 15:25:55 +00:00
Athanasius cd9c658286 Gateway: Don't use TLS cert in app setup
* The /upload/ functionality continues to work.
2022-03-12 15:14:48 +00:00
Athanasius f6dd3f21bc Settings: Remove {CERT,KEY}_FILE as first step to no more TLS 2022-03-12 15:11:15 +00:00
Athanasius 6b68a7c855 scripts/check-schemas-load: Standardise per-schema output
If we're going to say a schema is OK, then prefix any output with the
schema file location.
2022-03-12 15:07:42 +00:00
Athanasius c6e47813bb scripts/check-schemas-load: black-formatted, and be verbose 2022-03-12 15:05:10 +00:00
Athanasius 7523d6dc17 setup.py: Turn isort off/on around import setup_env.py
setup_env.py is NEVER checked into git, forcing anyone running the code
to ensure they have it set up correctly before setup.py will work.
2022-03-12 14:58:50 +00:00
Athanasius 32c4e83ff9 .flake8: Some tuning for EDDN specifically 2022-03-12 14:50:39 +00:00
Athanasius e74bc0b486 Gateway: Remove extraneous noqa's 2022-03-12 14:25:22 +00:00
Athanasius d059905f22 .python-version: Specify only 3.9 in general, not specific patch
We'll be using Debian bullseye's 3.9, which is currently 3.9.2
2022-03-12 14:22:17 +00:00
Athanasius ef909dc4d4 requirements-dev: Add "black" 2022-03-12 14:21:24 +00:00
Athanasius b838777d32 core/StatsCollector.py: black pass 2022-03-12 14:17:35 +00:00
Athanasius 47095d7615 core/Validator: black pass, and remove extraneous , in constants 2022-03-12 14:16:37 +00:00
Athanasius 32fb1b00ab core/DuplicateMessages: black / quotes pass 2022-03-12 14:14:31 +00:00
Athanasius 20d784ae09 setup.py: black and otherwise s/'/"/g pass 2022-03-12 14:10:41 +00:00
Athanasius 3cf7b765fb pyproject.toml: Exclude everything but actual core EDDN source 2022-03-12 14:07:42 +00:00
Athanasius 8da69a2101 conf/Settings: s/'/"/g; But not with black
We **do** want the special layout of the actual config defaults, so
screw black!
2022-03-12 13:48:57 +00:00
Athanasius 0e42fa71ea conf/__init__.py: docstring 2022-03-12 13:46:53 +00:00
Athanasius 3a9246a464 Relay: Re-format with black 2022-03-12 13:45:37 +00:00
Athanasius cf01daeaf2 Monitor: Re-format with black 2022-03-12 13:43:44 +00:00