schemas: Continue expanding on general README

This is pulling in a lot of what's currently in the wiki page, but
re-arranging and re-wording it.
This commit is contained in:
Athanasius 2021-10-04 16:25:18 +00:00
parent afa22f4784
commit dc4ae569ed

View File

@ -1,22 +1,28 @@
# EDDN Schemas Documentation
## Introduction
EDDN is a zermoq service to allow players of the game
[Elite Dangerous](https://www.elitedangerous.com/),
published by [Frontier Developments](https://www.frontier.co.uk/), to
upload game data so that interested listeners can receive a copy.
EDDN accepts HTTP POST uploads in a defined format representing this game
data and then passes it on to any interested listeners.
EDDN is a
[zermoq](https://zeromq.org/) service to allow players of the game
[Elite Dangerous](https://www.elitedangerous.com/), published
by [Frontier Developments](https://www.frontier.co.uk/), to upload game data so
that interested listeners can receive a copy.
EDDN accepts HTTP POST uploads in a defined format representing this game data
and then passes it on to any interested listeners.
---
## Sources
There are two sources of game data, both provided, and approved for use, by
Frontier Developments, the publisher of the game.
## Journal Files
On the PC version of the game "Journal files" are written during any game
session. These are in newline-delimited JSON format, with each line
representing a single JSON object. Frontier Developments publishes
### Journal Files
On the PC version of the game "Journal files" are written during any game
session. These are in newline-delimited JSON format, with each line
representing a single JSON object. Frontier Developments publishes
documentation for the various events in their
[Player Tools & API Discussions](https://forums.frontier.co.uk/forums/elite-api-and-tools/)
forum.
@ -28,23 +34,161 @@ In general the documentation is made available in a file named something like:
as both a MicroSoft word `.doc` file, or a `.pdf` file.
Consult the latest of these for documentation on individual events.
However, be aware that sometimes the documentation is in error, possibly
due to not having been updated after a game client change.
However, be aware that sometimes the documentation is in error, possibly due to
not having been updated after a game client change.
## Companion API (CAPI) data
Historically Frontier Developments provided an API for use by its
short-lived iOS "Companion" app. Initially this was only meant to be used
by that app, with no public documentation, or even admission of its existence.
### Companion API (CAPI) data
Initially it required being supplied with the email and password as used to
login to the game (but at least over HTTPS). Eventually they
[allowed general use of this](https://forums.frontier.co.uk/threads/open-letter-to-frontier-developments.218658/page-19#post-3371472).
Historically Frontier Developments provided an API for use by its short-lived
iOS "Companion" app. This was only intended to be used by that app, with no
public documentation, or even admission of its existence.
In late 2018 Frontier switched the authentication to using an oAuth2 flow,
meaning players no longer need to supply their email and password to
Eventually, after some enterprising players had snooped the connections and
figured out the login method and endpoints, Frontier Developments
[allowed general use of this](https://forums.frontier.co.uk/threads/open-letter-to-frontier-developments.218658/page-19#post-3371472)
.
Originally the API authentication required being supplied with the email and
password as used to login to the game (but at least this was over HTTPS).
In late 2018 Frontier switched the authentication to using an oAuth2 flow,
meaning players no longer need to supply their email and password to
third-party sites and clients.
As of October 2021 there has still never been any official documentation
about the available endpoints and how they work. There is some
As of October 2021 there has still never been any official documentation about
the available endpoints and how they work. There is some
[third-party documentation](https://github.com/Athanasius/fd-api/blob/main/docs/README.md)
by Athanasius.
When using the Companion API please be aware that the server that supplies this
data sometimes lags behind the game - usually by a few seconds, sometimes by
minutes. You MUST check in the data from the API that the Cmdr is
docked (`["commander"]["docked"]` is `True`) and that the station and
system (`["lastStarport"]["name"]` and `["lastSystem"]["name"]`) match those
reported from the Journal before using the data for the commodity, outfitting
and shipyard schemas.
---
## Uploading messages
To upload market data to EDDN, you'll need to make a POST request to the URL:
* https://eddn.edcd.io:4430/upload/
### Send only live data to the live schemas
You MUST NOT send information from any non-live (e.g. alpha or beta)
version of the game to the main schemas on this URL.
You MAY send such to this URL so long as you append `/test` to the `$schemaRef`
value, e.g.
"$schemaRef": "https://eddn.edcd.io/schemas/shipyard/2/test",
You can also utilise these test forms of the schemas when first testing your
code. There might also be a beta.eddn.edcd.io, or dev.eddn.edcd.io, service
available from time to time as necessary, e.g. for testing new schemas or
changes to existing ones.
### Format of uploaded messages
Each message is a JSON object in utf-8 encoding containing the following
key+value pairs:
1. `$schemaRef` - Which schema (and version) this message is for.
2. `header` - Object containing mandatory information about the upload;
1. `uploaderID` - a unique ID for the player uploading this data.
Don't worry about privacy, the EDDN service will hash this with a key
that is regularly changed so no-one knows who an uploader is in-game.
2. `softwareName` - an identifier for the software performing the upload.
3. `softwareVersion` - The version of that software being used.
Listeners MAY make decisions about whether to utilise the data in any
message based on the combination of `softwareName` and `softwareVersion`.
NB: **DO not** add `gatewaytimestamp` yourself. The EDDN Gateway will add
this and will overwrite any that you provide, so don't bother.
3. `message` - Object containing the data for this message. Consult the
relevant README file within this documentation, e.g.
[codexentry-README.md](./codexentry-README.md). There are some general
guidelines [below](#contents-of-message).
For example, a shipyard message, version 2, might look like:
```JSON
{
"$schemaRef": "https://eddn.edcd.io/schemas/shipyard/2",
"header": {
"uploaderID": "Bill",
"softwareName": "My excellent app",
"softwareVersion": "0.0.1"
},
"message": {
"systemName": "Munfayl",
"stationName": "Samson",
"marketId": 128023552,
"horizons": true,
"timestamp": "2019-01-08T06:39:43Z",
"ships": [
"anaconda",
"dolphin",
"eagle",
"ferdelance",
"hauler",
"krait_light",
"krait_mkii",
"mamba",
"python",
"sidewinder"
]
}
}
```
### Contents of `message`
Each `message` object must have, at bare minimum:
1. `timestamp` - string date and time in ISO8601 format. Whilst that
technically allows for any timezone to be cited you SHOULD provide this in
UTC, aka 'Zulu Time' as in the example above. You MUST ensure that you are
doing this properly. Do not claim 'Z' whilst actually using a local time
that is offset from UTC.
2. One other key/value pair representing the data. In general there will be
much more than this. Again, consult the
[schemas and their documentation](./).
EDDN is intended to transport generic data not specific to any particular Cmdr
and to reflect the data that a player would see in-game in station services or
the local map. To that end, uploading applications MUST ensure that messages do
not contain any Cmdr-specific data (other than "uploaderID" and the "horizons"
flag). In practice as of E:D 3.3 this means:
* commodity: Skip commodities with `"categoryname": "NonMarketable"` (i.e.
Limpets - not purchasable in station market) or `"legality":` *non-empty
string* (not normally traded at this station market).
* outfitting: Skip items whose availability depends on the Cmdr's status rather
than on the station. Namely:
- Items that aren't weapons/utilities (`Hpt_*`), standard/internal
modules (`Int_*`) or armour (`*_Armour_*`) (i.e. bobbleheads, decals,
paintjobs and shipkits).
- Items that have a non-null `"sku"` property, unless
it's `"ELITE_HORIZONS_V_PLANETARY_LANDINGS"` (i.e. PowerPlay and tech
broker items).
- The `"Int_PlanetApproachSuite"` module (for historical reasons).
* shipyard: *Include* ships listed in the `"unavailable_list"` property (i.e.
available at this station, but not to this Cmdr).
* journal: Strip out `"..._Localised"` properties throughout the data
structure.
* journal/Docked: Strip out `"Wanted"`, `"ActiveFine"`, `"CockpitBreach"`
properties
* journal/FSDJump: Strip out `"Wanted"`, `"BoostUsed"`, `"FuelLevel"`
, `"FuelUsed"` and `"JumpDist"` properties.
* journal/Location: Strip out `"Wanted"`, `"Latitude"` and `"Longitude"`
properties.
* journal/Location and journal/FSDJump: strip out `"HappiestSystem"`
, `"HomeSystem"`, `"MyReputation"` and `"SquadronFaction"` properties within
the list of `"Factions"`.
Some of these requirements are also enforced by the schemas, and some things
the schemas enforce might not be explicitly called out here, so **do**
check what you're sending against the schema when implementing sending new
events.