mirror of
https://github.com/EDCD/EDDN.git
synced 2025-06-19 16:23:56 +03:00
docs/Running: Specify network/firewall config, correct some paths and ...
* <hr> between sections * Document the ports that *must* be open to the internet. * Apache will require the proxy_http module. * Move the github clone earlier in instructions, as part of ensuring paths exist when they should. * Note about needing to do hostname and port substitutions in monitor files if not using the standard values. * Link to other sections where appropriate.
This commit is contained in:
parent
b1056c1239
commit
0bcb9cc545
@ -19,6 +19,8 @@ A specific user was created:
|
|||||||
|
|
||||||
useradd -c 'EDDN Gateway' -m -s /bin/bash eddn
|
useradd -c 'EDDN Gateway' -m -s /bin/bash eddn
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# Further installation
|
# Further installation
|
||||||
|
|
||||||
## As 'root'
|
## As 'root'
|
||||||
@ -38,6 +40,8 @@ You will need a mysql/mariab database:
|
|||||||
> GRANT ALL PRIVILEGES on eddn.* TO 'eddn'@'localhost';
|
> GRANT ALL PRIVILEGES on eddn.* TO 'eddn'@'localhost';
|
||||||
> \q
|
> \q
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
### Netdata
|
### Netdata
|
||||||
In order to get host performance metrics (CPU, RAM and network usage) you will
|
In order to get host performance metrics (CPU, RAM and network usage) you will
|
||||||
need to install netdata. On Debian-based systems:
|
need to install netdata. On Debian-based systems:
|
||||||
@ -47,6 +51,8 @@ need to install netdata. On Debian-based systems:
|
|||||||
The default configuration should be all you need, listening on
|
The default configuration should be all you need, listening on
|
||||||
`127.0.0.1:19999`.
|
`127.0.0.1:19999`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
### LetsEncrypt
|
### LetsEncrypt
|
||||||
We assume that you're using a TLS certificate from
|
We assume that you're using a TLS certificate from
|
||||||
[LetsEncrypt](https://letsencrypt.org/), it's free!
|
[LetsEncrypt](https://letsencrypt.org/), it's free!
|
||||||
@ -113,15 +119,63 @@ Debian system using certbot:
|
|||||||
|
|
||||||
**Remember to edit them to suit your setup!**
|
**Remember to edit them to suit your setup!**
|
||||||
|
|
||||||
### Reverse Proxy with Apache
|
---
|
||||||
|
|
||||||
|
### Network Configuration
|
||||||
|
There are multiple ports that you'll have to ensure are allowed through any
|
||||||
|
firewall, and some of them also require being reverse proxied correctly.
|
||||||
|
|
||||||
|
The reverse proxies pertain to:
|
||||||
|
|
||||||
|
1. The port for the Gateway to receive uploads from senders (e.g. Elite
|
||||||
|
Dangerous Market Connector). This is also used for the 'monitor' web
|
||||||
|
page to obtain stats about messages passing through the Gateway.
|
||||||
|
|
||||||
|
1. A set of URLs for accessing [netdata](#netdata).
|
||||||
|
|
||||||
|
#### Necessary ports
|
||||||
|
These all for TCP, no UDP:
|
||||||
|
|
||||||
|
1. `443` - a web server capable of reverse proxying set up for TLS on the
|
||||||
|
public host name of the EDDN service. This is used to serve the schemas,
|
||||||
|
the monitor web page, and to reverse proxy URLs beginning `/netdata/` to
|
||||||
|
the [netdata](#netdata) service.
|
||||||
|
|
||||||
|
1. Default: `4430` - Gateway 'http' port, used both for EDDN senders to
|
||||||
|
upload, and also for the Gateway message rate stats on the monitor web
|
||||||
|
page.
|
||||||
|
|
||||||
|
But that's the *public* port. The Gateway process itself listens on `8081`.
|
||||||
|
So you'll need a reverse proxy listening on port `4430` and forwarding
|
||||||
|
*all* requests to `127.0.0.1:8081`.
|
||||||
|
|
||||||
|
1. Default: `9091` - Monitor 'http' port, used for the monitor web page to
|
||||||
|
query schema and software statistics. No reverse proxy setup.
|
||||||
|
|
||||||
|
1. Default: `9500` - The port on the Relay that EDDN listeners connect to in
|
||||||
|
order to receive the zeromq stream. No reverse proxy setup.
|
||||||
|
|
||||||
|
1. Default: `9090` - The Relay 'http' port for its portion of the message
|
||||||
|
statistics on the monitor web page. No reverse proxy setup.
|
||||||
|
|
||||||
|
There's also the internal `8500` port, but that's literally only used for
|
||||||
|
the Monitor and Relay to pick up zeromq messages forwarded from the
|
||||||
|
Gateway, so all over localhost.
|
||||||
|
|
||||||
|
See [Configuration](#configuration) for guidance on what override config
|
||||||
|
settings can be used to change any of these ports.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Reverse Proxy with Apache
|
||||||
If you already have an Apache installation it will be easier to just use
|
If you already have an Apache installation it will be easier to just use
|
||||||
it for the reverse proxy.
|
it for the reverse proxy.
|
||||||
|
|
||||||
Ensure you have the module installed and active:
|
Ensure you have these modules installed and active:
|
||||||
|
|
||||||
a2enmod proxy
|
a2enmod proxy proxy_http
|
||||||
|
|
||||||
#### Apache configuration
|
##### Apache configuration
|
||||||
There is an example VirtualHost configuration in
|
There is an example VirtualHost configuration in
|
||||||
`contrib/apache-eddn.conf` which makes the following assumptions:
|
`contrib/apache-eddn.conf` which makes the following assumptions:
|
||||||
|
|
||||||
@ -145,12 +199,14 @@ You should be able to:
|
|||||||
# CHECK THE OUTPUT
|
# CHECK THE OUTPUT
|
||||||
apache2ctl graceful
|
apache2ctl graceful
|
||||||
|
|
||||||
### Reverse Proxy with nginx
|
---
|
||||||
|
|
||||||
|
#### Reverse Proxy with nginx
|
||||||
If you don't yet have nginx installed then start with:
|
If you don't yet have nginx installed then start with:
|
||||||
|
|
||||||
apt install nginx-light
|
apt install nginx-light
|
||||||
|
|
||||||
#### nginx configuration
|
##### nginx configuration
|
||||||
There is an example configuration in `contrib/nginx-eddn.conf` which makes
|
There is an example configuration in `contrib/nginx-eddn.conf` which makes
|
||||||
some assumptions:
|
some assumptions:
|
||||||
|
|
||||||
@ -175,18 +231,29 @@ If you're already using another web server you'll need to
|
|||||||
duplicate at least the use of a TLS certificate and the Reverse Proxying as
|
duplicate at least the use of a TLS certificate and the Reverse Proxying as
|
||||||
required.
|
required.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## In the 'eddn' account
|
## In the 'eddn' account
|
||||||
|
|
||||||
|
### Clone a copy of the application project from gitub
|
||||||
|
|
||||||
|
mkdir -p ${HOME}/dev
|
||||||
|
cd ${HOME}/dev
|
||||||
|
git clone https://github.com/EDCD/EDDN.git EDDN.git
|
||||||
|
cd EDDN.git
|
||||||
|
|
||||||
|
We'll assume this `${HOME}/dev/EDDN.git` path elsewhere in this document.
|
||||||
|
|
||||||
### Set up a python virtual environment
|
### Set up a python virtual environment
|
||||||
So as to not have any python package version requirements clash with
|
So as to not have any python package version requirements clash with
|
||||||
anything else it's best to use a Python virtual environment (venv). You
|
anything else it's best to use a Python virtual environment (venv). You
|
||||||
will have installed the Debian package 'virtualenv' above for this purpose.
|
will have installed the Debian package 'virtualenv' [above](#as-root) for
|
||||||
|
this purpose.
|
||||||
|
|
||||||
We'll put the venv in `${HOME}/eddn/python2.7-venv` with the following
|
We'll put the venv in `${HOME}/dev/python2.7-venv` with the following
|
||||||
command:
|
command:
|
||||||
|
|
||||||
mkdir -p ${HOME}/eddn
|
cd ${HOME}/dev
|
||||||
cd ${HOME}/eddn
|
|
||||||
virtualenv -p /usr/bin/python2.7 ${HOME}/python2.7-venv
|
virtualenv -p /usr/bin/python2.7 ${HOME}/python2.7-venv
|
||||||
|
|
||||||
And for future ease of changing python versions:
|
And for future ease of changing python versions:
|
||||||
@ -197,15 +264,6 @@ And now start using this venv:
|
|||||||
|
|
||||||
. python-venv/bin/activate
|
. python-venv/bin/activate
|
||||||
|
|
||||||
### Clone a copy of the application project from gitub
|
|
||||||
|
|
||||||
mkdir -p ${HOME}/eddn/dev
|
|
||||||
cd ${HOME}/eddn/dev
|
|
||||||
git clone https://github.com/EDCD/EDDN.git
|
|
||||||
cd EDDN
|
|
||||||
|
|
||||||
We'll assume this `${HOME}/eddn/dev/EDDN` path elsewhere in this document.
|
|
||||||
|
|
||||||
### Ensure necessary python modules are installed
|
### Ensure necessary python modules are installed
|
||||||
Installing extra necessary python modules is simple:
|
Installing extra necessary python modules is simple:
|
||||||
|
|
||||||
@ -217,6 +275,10 @@ You will need to get the database schema in place:
|
|||||||
mysql -p eddn < ${HOME}/eddn/dev/EDDN/schema.sql
|
mysql -p eddn < ${HOME}/eddn/dev/EDDN/schema.sql
|
||||||
<the password you set in the "CREATE USER" statement above>
|
<the password you set in the "CREATE USER" statement above>
|
||||||
|
|
||||||
|
Ref: [As root](#as-root).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# Concepts
|
# Concepts
|
||||||
There are three components to this application.
|
There are three components to this application.
|
||||||
|
|
||||||
@ -248,7 +310,11 @@ test host. The files in question are:
|
|||||||
monitor/js/eddn.js
|
monitor/js/eddn.js
|
||||||
monitor/schemas.html
|
monitor/schemas.html
|
||||||
|
|
||||||
Replace the string `eddn.edcd.io` with the hostname you're using.
|
Replace the string `eddn.edcd.io` with the hostname you're using. You'll need
|
||||||
|
to perform similar substitutions if you change the configuration to use any
|
||||||
|
different port numbers.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# Configuration
|
# Configuration
|
||||||
Default application configuration is in the file `src/eddn/conf/Settings.py`.
|
Default application configuration is in the file `src/eddn/conf/Settings.py`.
|
||||||
@ -258,36 +324,45 @@ another file.
|
|||||||
1. You will need to obtain a TLS certificate from, e.g. LetsEncrypt. The
|
1. You will need to obtain a TLS certificate from, e.g. LetsEncrypt. The
|
||||||
application will need access to this and its private key file.
|
application will need access to this and its private key file.
|
||||||
|
|
||||||
CERT_FILE = '/etc/letsencrypt/live/eddn.edcd.io/fullchain.pem'
|
CERT_FILE = '/etc/letsencrypt/live/YOUROWN.eddn.edcd.io/fullchain.pem'
|
||||||
KEY_FILE = '/etc/letsencrypt/live/eddn.edcd.io/privkey.pem'
|
KEY_FILE = '/etc/letsencrypt/live/YOUROWN.eddn.edcd.io/privkey.pem'
|
||||||
|
|
||||||
1. Network configuration
|
1. Network configuration
|
||||||
1. `RELAY_HTTP_BIND_ADDRESS` and `RELAY_HTTP_PORT` define the IP and port
|
1. `RELAY_HTTP_BIND_ADDRESS` and `RELAY_HTTP_PORT` define the IP and port
|
||||||
on which the Relay listens for, e.g. `/stats/` requests.
|
on which the Relay listens for, e.g. `/stats/` requests.
|
||||||
|
|
||||||
1. `RELAY_RECEIVER_BINDINGS` defines where the Relay connects in order to
|
1. `RELAY_RECEIVER_BINDINGS` defines where the Relay connects in order to
|
||||||
subscribe to messages from the Gateway. Should match
|
subscribe to messages from the Gateway. Should match
|
||||||
`GATEWAY_SENDER_BINDINGS`.
|
`GATEWAY_SENDER_BINDINGS`.
|
||||||
|
|
||||||
1. `RELAY_SENDER_BINDINGS` defines the address the application listens on
|
1. `RELAY_SENDER_BINDINGS` defines the address the application listens on
|
||||||
for connections from listeners such as eddb.io.
|
for connections from listeners such as eddb.io.
|
||||||
|
|
||||||
1. `RELAY_DUPLICATE_MAX_MINUTES` how many minutes to keep messages hashes
|
1. `RELAY_DUPLICATE_MAX_MINUTES` how many minutes to keep messages hashes
|
||||||
cached for so as to detect, and not Relay out, duplicate messages. If
|
cached for so as to detect, and not Relay out, duplicate messages. If
|
||||||
you set this to the literal string `false` the duplication checks will be
|
you set this to the literal string `false` the duplication checks will be
|
||||||
disabled. This is **very handy** when testing the code.
|
disabled. This is **very handy** when testing the code.
|
||||||
|
|
||||||
1. `GATEWAY_HTTP_BIND_ADDRESS` and `GATEWAY_HTTP_PORT` define where the
|
1. `GATEWAY_HTTP_BIND_ADDRESS` and `GATEWAY_HTTP_PORT` define where the
|
||||||
Gateway listens to for incoming messages from senders. Might be
|
Gateway listens to for incoming messages from senders. Might be
|
||||||
forwarded from nginx or other reverse proxy.
|
forwarded from nginx or other reverse proxy.
|
||||||
|
|
||||||
1. `GATEWAY_SENDER_BINDINGS` is where the Gateway listens for connections
|
1. `GATEWAY_SENDER_BINDINGS` is where the Gateway listens for connections
|
||||||
from the Relay and Monitor in order to send them messages that passed
|
from the Relay and Monitor in order to send them messages that passed
|
||||||
schema checks.
|
schema checks.
|
||||||
|
|
||||||
1. `GATEWAY_JSON_SCHEMAS` defines the schemas used for validation. Note
|
1. `GATEWAY_JSON_SCHEMAS` defines the schemas used for validation. Note
|
||||||
that these are full public URLs which are served by nginx (or whatever
|
that these are full public URLs which are served by your web server.
|
||||||
else you're using as the reverse proxy).
|
|
||||||
1. `GATEWAY_OUTDATED_SCHEMAS` any past schemas that are no longer valid.
|
1. `GATEWAY_OUTDATED_SCHEMAS` any past schemas that are no longer valid.
|
||||||
|
|
||||||
1. `MONITOR_HTTP_BIND_ADDRESS` and `MONITOR_HTTP_PORT` define where the
|
1. `MONITOR_HTTP_BIND_ADDRESS` and `MONITOR_HTTP_PORT` define where the
|
||||||
Monitor listens to for web connections, e.g. the statistics page.
|
Monitor listens to for web connections, e.g. the statistics page.
|
||||||
|
|
||||||
1. `MONITOR_RECEIVER_BINDINGS` defines where the Monitor connects in order
|
1. `MONITOR_RECEIVER_BINDINGS` defines where the Monitor connects in order
|
||||||
to subscribe to messages from the Gateway. Should match
|
to subscribe to messages from the Gateway. Should match
|
||||||
`GATEWAY_SENDER_BINDINGS`.
|
`GATEWAY_SENDER_BINDINGS`.
|
||||||
|
|
||||||
1. `MONITOR_UA` appears to be unused.
|
1. `MONITOR_UA` appears to be unused.
|
||||||
|
|
||||||
1. Database Configuration
|
1. Database Configuration
|
||||||
@ -295,8 +370,8 @@ another file.
|
|||||||
connect to a mysql/mariadb database for storing stats.
|
connect to a mysql/mariadb database for storing stats.
|
||||||
1. `database` - the name of the database
|
1. `database` - the name of the database
|
||||||
1. `user` - the user to connect as
|
1. `user` - the user to connect as
|
||||||
1. `password` - the secure password you set above when installing and
|
1. `password` - the secure password you set [above](#as-root) when
|
||||||
configuring mariadb/mysql.
|
installing and configuring mariadb/mysql.
|
||||||
|
|
||||||
It is assumed that the database is on `localhost`.
|
It is assumed that the database is on `localhost`.
|
||||||
|
|
||||||
@ -320,6 +395,8 @@ It sets:
|
|||||||
1. Configures the database connection and credentials.
|
1. Configures the database connection and credentials.
|
||||||
1. Turns off the relay duplicate check.
|
1. Turns off the relay duplicate check.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# Running
|
# Running
|
||||||
You have some choices for how to run the application components:
|
You have some choices for how to run the application components:
|
||||||
|
|
||||||
@ -344,6 +421,8 @@ You have some choices for how to run the application components:
|
|||||||
running the components. They will need the `DAEMON` lines tweaking for
|
running the components. They will need the `DAEMON` lines tweaking for
|
||||||
running from another location.
|
running from another location.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# Accessing the Monitor
|
# Accessing the Monitor
|
||||||
There is an EDDN Status web page usually provided at, e.g.
|
There is an EDDN Status web page usually provided at, e.g.
|
||||||
https://eddn.edcd.io/. This is enabled by the Monitor component through
|
https://eddn.edcd.io/. This is enabled by the Monitor component through
|
||||||
@ -365,7 +444,9 @@ permissions are correct for your web server to access them.
|
|||||||
chmod -R og+rX ${HOME} ${HOME}/.local ${HOME}/.local/share ${HOME}/.local/share/eddn
|
chmod -R og+rX ${HOME} ${HOME}/.local ${HOME}/.local/share ${HOME}/.local/share/eddn
|
||||||
chmod -R og+rX ${HOME}/.local/share/eddn/schemas
|
chmod -R og+rX ${HOME}/.local/share/eddn/schemas
|
||||||
|
|
||||||
## Testing all of this in a VM
|
---
|
||||||
|
|
||||||
|
# Testing all of this in a VM
|
||||||
In order to test all of this in a VM you might need to set up a double
|
In order to test all of this in a VM you might need to set up a double
|
||||||
proxying:
|
proxying:
|
||||||
|
|
||||||
|
Loading…
x
Reference in New Issue
Block a user