952 Commits

Author SHA1 Message Date
Athanasius
9a1cc7b141 Finally got the correct setup.py configuration for source of egg running
* py_modules parameter to setup() isn't documented in the setuptools
  docs, but is in a general 'python packaging' one.
* So now the main scripts are NOT within the `eddn` package..
* But all other code is...
* But the schema files don't need to be.
2022-03-14 14:51:11 +00:00
Athanasius
39058653cb More file re-arrangement, concentrating on running from source
With `import eddn.core.Validator` and the like it was actually picking
up the egg versions of the files, not local ones.

Currently this will run as per:

cd src/eddn
python Gateway.py --config ~/.local/share/eddn/dev/config.json
2022-03-14 14:21:22 +00:00
Athanasius
c402617760 Initial re-arrangement of files
* Move the service scripts out of src/eddn/ into src/
* Move the schemas/ to src/schemas/
* Some updates to setup.py, i.e. source of files, but the setuptools
  part isn't yet work.
2022-03-14 13:12:58 +00:00
Athanasius
0e8e22a146 contrib: apache-eddn.conf: Attempt to correct for new "no direct access"
As we've changed things to default to *not* doing TLS on the actual
services, and thus relying on reverse proxying of all the relevant
end points, this needed updating.

Not yet tested, that will come when I put this on dev.eddn.edcd.io.
2022-03-13 16:56:21 +00:00
Athanasius
bd3fdce53b monitor: Make example eddn-config.js fit the usual live setup 2022-03-13 16:46:11 +00:00
Athanasius
0916d0c440 monitor/schemas.html: Actually, those bits are schema URLs, so revert change
It's only the monitor endpoint URL that needed to be configurable.
Anyone using this wholesale for schemas not under eddn.edcd.io gets to
make the necessary hard-coded changes.
2022-03-13 16:41:04 +00:00
Athanasius
11ac577e34 monitor/schemas.html: Now supports eddn-config.js 2022-03-13 16:37:30 +00:00
Athanasius
6d4d449bb7 setup.py: Backup/restore monitor eddn-config.js if present 2022-03-13 16:33:56 +00:00
Athanasius
d953153d0b docs/Running: Update for monitor eddn-config.json
Although now I realise the schemas.html is going to need some work.
2022-03-13 16:07:08 +00:00
Athanasius
6e88a638a5 monitor: Move eddn-config.js to only an example, and make it such
We won't want the live config copied in/overwritten on each invocation
of 'python setup.py install'.
2022-03-13 15:57:41 +00:00
Athanasius
063322df4f monitor: Move configuration into separate file
This will allow for running more than one monitor without editing actual
source.
2022-03-13 15:55:03 +00:00
Athanasius
8c62afb869 Make per-component logging more obvious 2022-03-13 15:42:53 +00:00
Athanasius
52497dfc38 Settings: Default Relay and Monitor to localhost listen. 2022-03-13 15:34:54 +00:00
Athanasius
d0a98fe1b7 Relay: Allow for running without TLS 2022-03-13 15:34:09 +00:00
Athanasius
e8fa824b9d Monitor: Change to be able to run TLS-less
NB: Not yet changed the actual monitor web page files.
2022-03-13 13:59:15 +00:00
Athanasius
1166b814b5 TLS: Optionally use TLS if you set non-empty CERT_FILE and KEY_FILE
Whilst we do want to go TLS-less in the actual EDDN code, for ease of
setting up automated end to end functional testing, leave the
possibility of running with TLS termination as well.
2022-03-13 13:46:37 +00:00
Athanasius
80cae12a4c Gateway: Use a dict for kwargs to app.run()
This is so we can now adjust if we're putting the TLS cert/key files in.
2022-03-13 13:32:13 +00:00
Athanasius
82ad3d54c7 Gateway: Put remote_addr in [], as it could be multiple, comma-separated 2022-03-12 19:02:34 +00:00
Athanasius
b137d8c234 EDDNWSGIHandler: X-Forwarded-For can be a comma-separated list
So, make it easier to pull out the IPs, single or not.
2022-03-12 19:00:07 +00:00
Athanasius
c415e7c52a Move EDDNWSGIHandler into its own file 2022-03-12 17:12:17 +00:00
Athanasius
4d2512750a Gateway: Fixed logging to be consistent and use client IP
* Send all the bottle server output through our logger.
* Ensure gevent uses client IP, not 127.0.0.1.
2022-03-12 17:06:37 +00:00
Athanasius
d7e4425b1d Update docs and apache contrib file for 'no more TLS' 2022-03-12 15:25:55 +00:00
Athanasius
cd9c658286 Gateway: Don't use TLS cert in app setup
* The /upload/ functionality continues to work.
2022-03-12 15:14:48 +00:00
Athanasius
f6dd3f21bc Settings: Remove {CERT,KEY}_FILE as first step to no more TLS 2022-03-12 15:11:15 +00:00
Athanasius
6b68a7c855 scripts/check-schemas-load: Standardise per-schema output
If we're going to say a schema is OK, then prefix any output with the
schema file location.
2022-03-12 15:07:42 +00:00
Athanasius
c6e47813bb scripts/check-schemas-load: black-formatted, and be verbose 2022-03-12 15:05:10 +00:00
Athanasius
7523d6dc17 setup.py: Turn isort off/on around import setup_env.py
setup_env.py is NEVER checked into git, forcing anyone running the code
to ensure they have it set up correctly before setup.py will work.
2022-03-12 14:58:50 +00:00
Athanasius
32c4e83ff9 .flake8: Some tuning for EDDN specifically 2022-03-12 14:50:39 +00:00
Athanasius
e74bc0b486 Gateway: Remove extraneous noqa's 2022-03-12 14:25:22 +00:00
Athanasius
d059905f22 .python-version: Specify only 3.9 in general, not specific patch
We'll be using Debian bullseye's 3.9, which is currently 3.9.2
2022-03-12 14:22:17 +00:00
Athanasius
ef909dc4d4 requirements-dev: Add "black" 2022-03-12 14:21:24 +00:00
Athanasius
b838777d32 core/StatsCollector.py: black pass 2022-03-12 14:17:35 +00:00
Athanasius
47095d7615 core/Validator: black pass, and remove extraneous , in constants 2022-03-12 14:16:37 +00:00
Athanasius
32fb1b00ab core/DuplicateMessages: black / quotes pass 2022-03-12 14:14:31 +00:00
Athanasius
20d784ae09 setup.py: black and otherwise s/'/"/g pass 2022-03-12 14:10:41 +00:00
Athanasius
3cf7b765fb pyproject.toml: Exclude everything but actual core EDDN source 2022-03-12 14:07:42 +00:00
Athanasius
8da69a2101 conf/Settings: s/'/"/g; But not with black
We **do** want the special layout of the actual config defaults, so
screw black!
2022-03-12 13:48:57 +00:00
Athanasius
0e42fa71ea conf/__init__.py: docstring 2022-03-12 13:46:53 +00:00
Athanasius
3a9246a464 Relay: Re-format with black 2022-03-12 13:45:37 +00:00
Athanasius
cf01daeaf2 Monitor: Re-format with black 2022-03-12 13:43:44 +00:00
Athanasius
e6ea9cb7a2 Bouncer: Re-format with black. 2022-03-12 13:41:27 +00:00
Athanasius
3ad3262542 pyproject.toml: Add file so as to configure black line-length 2022-03-12 13:01:33 +00:00
Athanasius
3683890ed0 Gateway: Format with black 2022-03-12 12:59:16 +00:00
Athanasius
a08fc09586 Gateway: Use "" for strings throughout 2022-03-12 12:52:48 +00:00
Athanasius
1e39ea37af src/eddn/core/__init__.py: docstring 2022-03-12 12:46:16 +00:00
Athanasius
e6911d2c04 Bouncer: Use "" for strings 2022-03-12 12:46:04 +00:00
Athanasius
1b431d5dcd conf/Version: Bump to 3.0 2022-03-12 12:39:44 +00:00
Athanasius
0a295d4540 conf/Settings: flake8 and mypy pass 2022-03-12 12:39:11 +00:00
Athanasius
a36fe8ddd6 Relay: flake8 and mypy pass 2022-03-12 12:37:35 +00:00
Athanasius
8a2d76849b Monitor: flake8 and mypy pass 2022-03-12 12:30:13 +00:00