969 Commits

Author SHA1 Message Date
Athanasius
804b0f3bcd requirements.txt: Bump all to latest available on pypi
Specifically pyzmq==22.3.0 had no Python 3.11 versions, so tried to
build from source, but *that* ran into issues with that version.
2022-12-21 16:59:06 +00:00
Athanasius
e1c33fb563 Relay: Add 'are you running this correctly?' check 2022-03-18 14:37:40 +00:00
Athanasius
3080c1f596 Monitor: Add 'are you running this correctly?' check 2022-03-18 14:25:19 +00:00
Athanasius
94e705a18a Bouncer: Add 'are you running this correctly?' check 2022-03-18 14:22:55 +00:00
Athanasius
6eedf6f4bf Gateway: Add 'are we running this correctly?' check
Also some misc import order fixups
2022-03-18 14:21:57 +00:00
Athanasius
63cd78327f pre-commit: Specify more flake8 dependencies 2022-03-18 14:20:25 +00:00
Athanasius
fc853dcecf pre-commit: Specify flake8-isort dependency 2022-03-18 14:19:19 +00:00
Athanasius
56ab2d9445 pre-commit: Change flake8 to repo-based 2022-03-18 14:18:09 +00:00
Athanasius
07ab15102e pre-commit: Additional mypy dependencies & update versions 2022-03-18 14:04:24 +00:00
Athanasius
ce69f2ef4b requirements: Sanitised with 'pip-chill' and bumped to 'eager' versions 2022-03-18 12:19:52 +00:00
Athanasius
5b110fa554 Add helper scripts for running from source, and document them. 2022-03-18 11:56:56 +00:00
Athanasius
f8872c6dfa Move main scripts back into src/eddn/
So, yes, running any of these absolutely directly:

    cd src
    python eddn/Gateway.py

will fail.  But you *can*:

    cd src
    python -m eddn.Gateway

and they work perfectly fine.
2022-03-18 11:41:51 +00:00
Athanasius
5f64fb1c33 docs: General update for switching to wheel for install 2022-03-14 17:03:30 +00:00
Athanasius
c6e5820683 docs/Running: Assume python3 for virtualenv 2022-03-14 16:53:18 +00:00
Athanasius
f5879fbce7 Rename main scripts to have "EDDN_" prefix
This avoids having possibly clashing files in the environment
sites-packages/, i.e. 'Gateway.py' is a bit too generic.
2022-03-14 16:17:22 +00:00
Athanasius
8345675b89 All change yet again: We're now building and installing a wheel, not an egg 2022-03-14 16:05:09 +00:00
Athanasius
cb5766a0eb setup.py: Remove comment about pinning module version, as we're not
And we're on python3 now anyway.
2022-03-14 14:55:16 +00:00
Athanasius
9a1cc7b141 Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.
2022-03-14 14:51:11 +00:00
Athanasius
39058653cb More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json
2022-03-14 14:21:22 +00:00
Athanasius
c402617760 Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.
2022-03-14 13:12:58 +00:00
Athanasius
0e8e22a146 contrib: apache-eddn.conf: Attempt to correct for new "no direct access"
As we've changed things to default to *not* doing TLS on the actual
services, and thus relying on reverse proxying of all the relevant
end points, this needed updating.

Not yet tested, that will come when I put this on dev.eddn.edcd.io.
2022-03-13 16:56:21 +00:00
Athanasius
bd3fdce53b monitor: Make example eddn-config.js fit the usual live setup 2022-03-13 16:46:11 +00:00
Athanasius
0916d0c440 monitor/schemas.html: Actually, those bits are schema URLs, so revert change
It's only the monitor endpoint URL that needed to be configurable.
Anyone using this wholesale for schemas not under eddn.edcd.io gets to
make the necessary hard-coded changes.
2022-03-13 16:41:04 +00:00
Athanasius
11ac577e34 monitor/schemas.html: Now supports eddn-config.js 2022-03-13 16:37:30 +00:00
Athanasius
6d4d449bb7 setup.py: Backup/restore monitor eddn-config.js if present 2022-03-13 16:33:56 +00:00
Athanasius
d953153d0b docs/Running: Update for monitor eddn-config.json
Although now I realise the schemas.html is going to need some work.
2022-03-13 16:07:08 +00:00
Athanasius
6e88a638a5 monitor: Move eddn-config.js to only an example, and make it such
We won't want the live config copied in/overwritten on each invocation
of 'python setup.py install'.
2022-03-13 15:57:41 +00:00
Athanasius
063322df4f monitor: Move configuration into separate file
This will allow for running more than one monitor without editing actual
source.
2022-03-13 15:55:03 +00:00
Athanasius
8c62afb869 Make per-component logging more obvious 2022-03-13 15:42:53 +00:00
Athanasius
52497dfc38 Settings: Default Relay and Monitor to localhost listen. 2022-03-13 15:34:54 +00:00
Athanasius
d0a98fe1b7 Relay: Allow for running without TLS 2022-03-13 15:34:09 +00:00
Athanasius
e8fa824b9d Monitor: Change to be able to run TLS-less
NB: Not yet changed the actual monitor web page files.
2022-03-13 13:59:15 +00:00
Athanasius
1166b814b5 TLS: Optionally use TLS if you set non-empty CERT_FILE and KEY_FILE
Whilst we do want to go TLS-less in the actual EDDN code, for ease of
setting up automated end to end functional testing, leave the
possibility of running with TLS termination as well.
2022-03-13 13:46:37 +00:00
Athanasius
80cae12a4c Gateway: Use a dict for kwargs to app.run()
This is so we can now adjust if we're putting the TLS cert/key files in.
2022-03-13 13:32:13 +00:00
Athanasius
82ad3d54c7 Gateway: Put remote_addr in [], as it could be multiple, comma-separated 2022-03-12 19:02:34 +00:00
Athanasius
b137d8c234 EDDNWSGIHandler: X-Forwarded-For can be a comma-separated list
So, make it easier to pull out the IPs, single or not.
2022-03-12 19:00:07 +00:00
Athanasius
c415e7c52a Move EDDNWSGIHandler into its own file 2022-03-12 17:12:17 +00:00
Athanasius
4d2512750a Gateway: Fixed logging to be consistent and use client IP
* Send all the bottle server output through our logger.
* Ensure gevent uses client IP, not 127.0.0.1.
2022-03-12 17:06:37 +00:00
Athanasius
d7e4425b1d Update docs and apache contrib file for 'no more TLS' 2022-03-12 15:25:55 +00:00
Athanasius
cd9c658286 Gateway: Don't use TLS cert in app setup
* The /upload/ functionality continues to work.
2022-03-12 15:14:48 +00:00
Athanasius
f6dd3f21bc Settings: Remove {CERT,KEY}_FILE as first step to no more TLS 2022-03-12 15:11:15 +00:00
Athanasius
6b68a7c855 scripts/check-schemas-load: Standardise per-schema output
If we're going to say a schema is OK, then prefix any output with the
schema file location.
2022-03-12 15:07:42 +00:00
Athanasius
c6e47813bb scripts/check-schemas-load: black-formatted, and be verbose 2022-03-12 15:05:10 +00:00
Athanasius
7523d6dc17 setup.py: Turn isort off/on around import setup_env.py
setup_env.py is NEVER checked into git, forcing anyone running the code
to ensure they have it set up correctly before setup.py will work.
2022-03-12 14:58:50 +00:00
Athanasius
32c4e83ff9 .flake8: Some tuning for EDDN specifically 2022-03-12 14:50:39 +00:00
Athanasius
e74bc0b486 Gateway: Remove extraneous noqa's 2022-03-12 14:25:22 +00:00
Athanasius
d059905f22 .python-version: Specify only 3.9 in general, not specific patch
We'll be using Debian bullseye's 3.9, which is currently 3.9.2
2022-03-12 14:22:17 +00:00
Athanasius
ef909dc4d4 requirements-dev: Add "black" 2022-03-12 14:21:24 +00:00
Athanasius
b838777d32 core/StatsCollector.py: black pass 2022-03-12 14:17:35 +00:00
Athanasius
47095d7615 core/Validator: black pass, and remove extraneous , in constants 2022-03-12 14:16:37 +00:00