* This did, however, remind me that the `data` passed into `cmdr_data()`
is an amalgam of `/profile`, `/market` and `/shipyard` queries.
This means that the data.source_endpoint is **not correct for all of
the data and its use**. As such I had to pass 'hard coded' values into
the function from the various CAPI export functions. They know what it
is they're exporting.
* As this reminded me that "CAPI `data` is actually a `CAPIDATA`", I've
documented that in PLUGINS.md, but with a dire warning against relying on
any of the extra properties.
* In theory we would always see `Fileheader` and clear `pending[]`, but let's
be extra paranoid and also clear it if there's a gameversion/build difference
between the prior event and the current one.
1. Due to the _TIMEOUT on the actual `post()` of a message it would be
possible for new entries to get queued in the meantime. These queued
entries could be 'in session' and end up going through pending and thus
sent before one of the 'new session' events is detected so as to clear
pending. The `this.gameversion/build` could have changed in the meantime,
so are no longer correct if game client changed.
2. So, pass in the current gameversion/build when a message is pushed into
the queue, and parse those back out when they're pulled out of the queue.
3. Use those versions in the message, not `this.` versions.
* Record the 'state' version of these in `this`.
* Use those when constructing the message.
* NB: Need to check if messages can be retained in the queue across client
changes. Coming up ....
* Adds `monitor.is_live_galaxy()` for general use.
* Assumes Update 14 starts after 2022-11-29T09:00:00+00:00. That's the
currently schedule day, and recently the servers have been down by the time.
Likelihood of them coming back *up* quickly seems slim to none.
* If we couldn't parse the `gameversion` from Journal using
`semantic_version.Version.coerce()` this will fail, and assume we're on
the Legacy galaxy.
Whilst setting it to the same "CAPI-<endpoint>" string as `gameversion` in
these cases would probably be OK, that's not the intent of the EDDN
documentation, which has now been clarified.
* In case of apparent issues, have a `--trace-on` to better see what's (not)
happening.
All the old DBEUG logging, even if commented out, is now under this.
* Also added some INFO level logging for the legacy replay.jsonl conversion,
as it should be one-time per user.
* Some additional DEBUG logging for closing down.
* An aborted attempt was made to use a thread worker, but:
1. sqlite3 doesn't allow cross-thread use of the same sqlite3 connection.
2. Having an on-going query on one cursor, e.g. gathering all the
outstanding message `id`, whilst trying to DELETE a row hits a
"database is locked" error.
* So, back to tk `after()`. `send_message_by_id()` has been audited to ensure
its boolean return is accurate. So there shouldn't be any way in which to
get hung up on a single message *other than if the EDDN Gateway is having
issues, and thus it should be retried anyway*. Any reason for a 'bad
message' will cause `True` return and thus deletion of the message in
*this* call to `queue_check_and_send()`.
* There is a new `reschedule` parameter to `queue_check_and_send()`. If
`True` then at the end it should re-schedule.
There is a check in `journal_entry()` for the `Docked` event, and if this
occurs it will schedule `queue_check_and_send()` with `reschedule` set to
`False` so that we don't end up with multiple parallel schedulings.
It's still possible for a docking to have coincided with a scheduled run
and thus cause double-rate sending to EDDN, but we can live with that.
* The same scheduling mechanism is used, with a much smaller delay, to
process more than one queued message per run.
Hence the `have_rescheduled` bool *in* the function to indicate if a 'fast'
reschedule has already been set. This prevents the slow one *also* being
set in this scenario. The latter will be scheduled when the fast one
found no more rows to process.
* Erroneously used 'CAPI-commoodity' when it's 'CAPI-market' (name of the
CAPI endpoint, not anything to do with EDDN schema names, and '-commodity'
would also be wrong for that).
* Set `header` for (CAPI) `export_commodities()`.
* The eddn parts of the OUT_EDDN_DO_NOT_DELAY -> OUT_EDDN_DELAY change. This
includes the 'sense' of it being inverted from what it was.
* EDDN.REPLAY_DELAY is now a float, as it's used with `time.sleep()`. *This*
is the 400ms value for inter-message cooldown.
* EDDN.REPLAY_PERIOD is still an int, used with tk `after()`. This is how
often we attempt the queue.
* EDDN.session is no longer a thing, move that part of EDDN.close() to
EDDNSender.close().
* EDDN queue DB has `id`, not `message_id`.
* Now *looping* in the queue sender, not only sending the oldest message.
* Now that we're not trying to do "did we just/are we know docked?" in this
code it turns out that both CAPI and Journal messages can use the same
function for this.
* And as it's no longer journal-specific `EDDN.export_journal_entry()` has
been renamed to `EDDN.send_message()`.
This whole branch now needs to actually implement sending queued messages
when docked, and periodically in the case of initial failures.
* `EDDN.sendreplay()` is no longer used.
* In `prefsvarschanged()` there was a reference to `eddn.replayfile`, so as
to grey out the "Delay sending..." option if the file wasn't available.
So that's moot and also removed, but also...
* Comment the purpose of that line in `prefsvarchanged()` because it's not
immediately obvious.
In some cases the check might already have been done, but if not then this
is the last easy place to perform it.
NB: Unlike the old code this does *not* attempt to check "are we docked
*now* ?" for triggering sending of previously queue messages. That's
going to need a thread worker.
It's easier to check "should we send this message at all?" earlier. Currently
all of the following ('station data') do so:
* CAPI commodity, outfitting (also fcmaterials) and shipyard.
* Journal commodity, fcmaterials, outfitting, and shipyard.
This flag controls whether commodity, outfitting or shipyard schema messages
are sent. Thus 'MKT' ('market') is misleading. Rename it so the intent when
used is clear.
* This was perhaps originally meant for what the UI option says, i.e. "send
system and scan data", but is actually being used for anything that is
**NOT** 'station data' (even though *that* option has 'MKT' it includes
outfitting and shipyard as well).
So, just name this more sanely such that code using it is more obvious as
to the actual intent.
The sense of this `output` flag has been inverted (always?) for a long time.
1. I have the option "Delay sending until docked" showing as *off* in the UI.
2. My config.output value is `100000000001`.
3. The value of this flag is `4096`, which means 12th bit (starting from 1, not
zero).
4. So I have the bit set, but the option visibly off.
So, rename this both to be more pertinent to its use *and* to be correct as to
what `True` for it means.
