mirror of
https://github.com/EDCD/EDDN.git
synced 2025-04-23 20:10:29 +03:00
Merge branch 'develop'
This commit is contained in:
Athanasius
commit
87b33e4d1b
208
schemas/README-EDDN-schemas.md
Normal file
208
schemas/README-EDDN-schemas.md
Normal file
@ -0,0 +1,208 @@
|
||||
# EDDN Schemas Documentation
|
||||
|
||||
## Introduction
|
||||
|
||||
EDDN is a
|
||||
[zermoq](https://zeromq.org/) service to allow players of the game
|
||||
[Elite Dangerous](https://www.elitedangerous.com/), published
|
||||
by [Frontier Developments](https://www.frontier.co.uk/), to upload game data so
|
||||
that interested listeners can receive a copy.
|
||||
|
||||
EDDN accepts HTTP POST uploads in a defined format representing this game data
|
||||
and then passes it on to any interested listeners.
|
||||
|
||||
---
|
||||
## Sources
|
||||
|
||||
There are two sources of game data, both provided by the publisher of the game,
|
||||
Frontier Developerments. They are both explicitly approved for use by
|
||||
third-party software.
|
||||
|
||||
### Journal Files
|
||||
|
||||
On the PC version of the game, "Journal files" are written during any game
|
||||
session. These are in newline-delimited JSON format, with each line
|
||||
representing a single JSON object. Frontier Developments publishes
|
||||
documentation for the various events in their
|
||||
[Player Tools & API Discussions](https://forums.frontier.co.uk/forums/elite-api-and-tools/)
|
||||
forum.
|
||||
|
||||
In general the documentation is made available in a file named something like:
|
||||
|
||||
Journal_Manual-v<version>
|
||||
|
||||
as both a MicroSoft word `.doc` file, or a `.pdf` file. Historically the
|
||||
use of `_` versus `-` in those filenames has varied.
|
||||
|
||||
Consult the latest of these for documentation on individual events.
|
||||
However, be aware that sometimes the documentation is in error, possibly due to
|
||||
not having been updated after a game client change.
|
||||
|
||||
### Companion API (CAPI) data
|
||||
|
||||
Frontier Developments provides an API to retrieve certain game data, even
|
||||
without the game running. Historically this was for use by its short-lived
|
||||
iOS "Companion" app, and was only intended to be used by that app. There was no
|
||||
public documentation, or even admission of its existence.
|
||||
|
||||
Eventually, after some enterprising players had snooped the connections and
|
||||
figured out the login method and endpoints, Frontier Developments
|
||||
[allowed general use of this](https://forums.frontier.co.uk/threads/open-letter-to-frontier-developments.218658/page-19#post-3371472)
|
||||
.
|
||||
|
||||
Originally the API authentication required being supplied with the email and
|
||||
password as used to login to the game (but at least this was over HTTPS).
|
||||
|
||||
In late 2018 Frontier switched the authentication to using an oAuth2 flow,
|
||||
meaning players no longer need to supply their email and password to
|
||||
third-party sites and clients.
|
||||
|
||||
As of October 2021 there has still never been any official documentation about
|
||||
the available endpoints and how they work. There is some
|
||||
[third-party documentation](https://github.com/Athanasius/fd-api/blob/main/docs/README.md)
|
||||
by Athanasius.
|
||||
|
||||
When using the Companion API please be aware that the server that supplies this
|
||||
data sometimes lags behind the game - usually by a few seconds, sometimes by
|
||||
minutes. You MUST check in the data from the CAPI that the Cmdr is
|
||||
docked (`["commander"]["docked"]` is `True`) and that the station and
|
||||
system (`["lastStarport"]["name"]` and `["lastSystem"]["name"]`) match those
|
||||
reported from the Journal before using the data for the commodity, outfitting
|
||||
and shipyard schemas.
|
||||
|
||||
---
|
||||
|
||||
## Uploading messages
|
||||
|
||||
### Send only live data to the live schemas
|
||||
You MUST **NOT** send information from any non-live (e.g. alpha or beta)
|
||||
version of the game to the main schemas on this URL.
|
||||
|
||||
You MAY send such to this URL so long as you append `/test` to the `$schemaRef`
|
||||
value, e.g.
|
||||
|
||||
"$schemaRef": "https://eddn.edcd.io/schemas/shipyard/2/test",
|
||||
|
||||
You MUST also utilise these test forms of the schemas when first testing your
|
||||
code. There might also be a beta.eddn.edcd.io, or dev.eddn.edcd.io, service
|
||||
available from time to time as necessary, e.g. for testing new schemas or
|
||||
changes to existing ones.
|
||||
|
||||
### Sending data
|
||||
To upload market data to EDDN, you'll need to make a POST request to the URL:
|
||||
|
||||
* https://eddn.edcd.io:4430/upload/
|
||||
|
||||
The body of this is a JSON object, so you SHOULD set a `Content-Type` header of
|
||||
`applicaton/json`, and NOT any of:
|
||||
|
||||
* `application/x-www-form-urlencoded`
|
||||
* `multipart/form-data`
|
||||
* `text/plain`
|
||||
|
||||
### Format of uploaded messages
|
||||
Each message is a JSON object in utf-8 encoding containing the following
|
||||
key+value pairs:
|
||||
|
||||
1. `$schemaRef` - Which schema (including version) this message is for.
|
||||
2. `header` - Object containing mandatory information about the upload;
|
||||
1. `uploaderID` - a unique ID for the player uploading this data.
|
||||
Don't worry about privacy, the EDDN service will hash this with a key
|
||||
that is regularly changed so no-one knows who an uploader is in-game.
|
||||
2. `softwareName` - an identifier for the software performing the upload.
|
||||
3. `softwareVersion` - The version of that software being used.
|
||||
|
||||
Listeners MAY make decisions about whether to utilise the data in any
|
||||
message based on the combination of `softwareName` and `softwareVersion`.
|
||||
|
||||
**DO not** add `gatewaytimestamp` yourself. The EDDN Gateway will add
|
||||
this and will overwrite any that you provide, so don't bother.
|
||||
4. `message` - Object containing the data for this message. Consult the
|
||||
relevant README file within this documentation, e.g.
|
||||
[codexentry-README.md](./codexentry-README.md). There are some general
|
||||
guidelines [below](#contents-of-message).
|
||||
|
||||
For example, a shipyard message, version 2, might look like:
|
||||
|
||||
```JSON
|
||||
{
|
||||
"$schemaRef": "https://eddn.edcd.io/schemas/shipyard/2",
|
||||
"header": {
|
||||
"uploaderID": "Bill",
|
||||
"softwareName": "My excellent app",
|
||||
"softwareVersion": "0.0.1"
|
||||
},
|
||||
"message": {
|
||||
"systemName": "Munfayl",
|
||||
"stationName": "Samson",
|
||||
"marketId": 128023552,
|
||||
"horizons": true,
|
||||
"timestamp": "2019-01-08T06:39:43Z",
|
||||
"ships": [
|
||||
"anaconda",
|
||||
"dolphin",
|
||||
"eagle",
|
||||
"ferdelance",
|
||||
"hauler",
|
||||
"krait_light",
|
||||
"krait_mkii",
|
||||
"mamba",
|
||||
"python",
|
||||
"sidewinder"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Contents of `message`
|
||||
|
||||
Each `message` object must have, at bare minimum:
|
||||
|
||||
1. `timestamp` - string date and time in ISO8601 format. Whilst that
|
||||
technically allows for any timezone to be cited you SHOULD provide this in
|
||||
UTC, aka 'Zulu Time' as in the example above. You MUST ensure that you are
|
||||
doing this properly. Do not claim 'Z' whilst actually using a local time
|
||||
that is offset from UTC.
|
||||
|
||||
Listeners MAY make decisions on accepting data based on this time stamp,
|
||||
i.e. "too old".
|
||||
2. One other key/value pair representing the data. In general there will be
|
||||
much more than this. Again, consult the
|
||||
[schemas and their documentation](./).
|
||||
|
||||
EDDN is intended to transport generic data not specific to any particular Cmdr
|
||||
and to reflect the data that a player would see in-game in station services or
|
||||
the local map. To that end, uploading applications MUST ensure that messages do
|
||||
not contain any Cmdr-specific data (other than "uploaderID" and the "horizons"
|
||||
flag). In practice as of E:D 3.3 this means:
|
||||
|
||||
* commodity: Skip commodities with `"categoryname": "NonMarketable"` (i.e.
|
||||
Limpets - not purchasable in station market) or `"legality":` *non-empty
|
||||
string* (not normally traded at this station market).
|
||||
* outfitting: Skip items whose availability depends on the Cmdr's status rather
|
||||
than on the station. Namely:
|
||||
- Items that aren't weapons/utilities (`Hpt_*`), standard/internal
|
||||
modules (`Int_*`) or armour (`*_Armour_*`) (i.e. bobbleheads, decals,
|
||||
paintjobs and shipkits).
|
||||
- Items that have a non-null `"sku"` property, unless
|
||||
it's `"ELITE_HORIZONS_V_PLANETARY_LANDINGS"` (i.e. PowerPlay and tech
|
||||
broker items).
|
||||
- The `"Int_PlanetApproachSuite"` module (for historical reasons).
|
||||
* shipyard: *Include* ships listed in the `"unavailable_list"` property (i.e.
|
||||
available at this station, but not to this Cmdr).
|
||||
* journal: Strip out `"..._Localised"` properties throughout the data
|
||||
structure.
|
||||
* journal/Docked: Strip out `"Wanted"`, `"ActiveFine"`, `"CockpitBreach"`
|
||||
properties
|
||||
* journal/FSDJump: Strip out `"Wanted"`, `"BoostUsed"`, `"FuelLevel"`
|
||||
, `"FuelUsed"` and `"JumpDist"` properties.
|
||||
* journal/Location: Strip out `"Wanted"`, `"Latitude"` and `"Longitude"`
|
||||
properties.
|
||||
* journal/Location and journal/FSDJump: strip out `"HappiestSystem"`
|
||||
, `"HomeSystem"`, `"MyReputation"` and `"SquadronFaction"` properties within
|
||||
the list of `"Factions"`.
|
||||
|
||||
Some of these requirements are also enforced by the schemas, and some things
|
||||
the schemas enforce might not be explicitly called out here, so **do**
|
||||
check what you're sending against the schema when implementing sending new
|
||||
events.
|
||||
118
schemas/codexentry-README.md
Normal file
118
schemas/codexentry-README.md
Normal file
@ -0,0 +1,118 @@
|
||||
# EDDN CodexEntry Schema
|
||||
|
||||
## Introduction
|
||||
Here we document how to take data from an ED `CodexEntry` Journal Event and
|
||||
properly structure it for sending to EDDN.
|
||||
|
||||
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
|
||||
documentation for a schema such as this.
|
||||
|
||||
## Senders
|
||||
The primary data source for this schema is the ED Journal event `CodexEntry`.
|
||||
|
||||
### Elisions
|
||||
You MUST remove any key where the key name ends in
|
||||
`_Localised`.
|
||||
|
||||
You MUST remove the two keys `IsNewEntry` and `NewTraitsDiscovered`.
|
||||
|
||||
### Augmentations
|
||||
#### StarPos
|
||||
You MUST **add** a `StarPos` key with value of type `array` containing the
|
||||
galaxy co-ordinates of the system. You will need to have obtained these
|
||||
from prior event(s) upon the player arriving, or logging into, the system.
|
||||
|
||||
e.g. if the system is `Alpha Centauri`:
|
||||
```json
|
||||
"StarPos": [3.03125, -0.09375, 3.15625]
|
||||
```
|
||||
|
||||
#### BodyID and BodyName
|
||||
You SHOULD attempt to track the BodyName and BodyID where the player is
|
||||
and add keys/values for these.
|
||||
|
||||
You MUST track `BodyName` both from Status.json *and* also from some
|
||||
[Journal](./README-EDDN-schemas.md#journal-files)
|
||||
events in order to cross-check it before using the `BodyID` from
|
||||
[Journal](./README-EDDN-schemas.md#journal-files) events.
|
||||
|
||||
The following is correct as of game version 4.0.0.801 (Odyssey initial
|
||||
release, Update 7, plus one patch).
|
||||
|
||||
1. Record `journal_body_name` and `journal_body_id` from the
|
||||
`BodyName` and `BodyID` values in `ApproachBody` events.
|
||||
|
||||
This will occur when the player flies below Orbital Cruise altitude
|
||||
around a body.
|
||||
2. Also record these from `Location` events to cover logging in already there.
|
||||
3. Unset both `journal_body_name` and `journal_body_id` on `LeaveBody` and
|
||||
`FSDJump` events.
|
||||
Do NOT do so for `SupercruiseEntry`, because a player can enter supercruise
|
||||
below max Orbital Cruise altitude and then come back down without a new
|
||||
`ApproachBody` event occurring.
|
||||
4. If Status.json has `BodyName` present, record that as `status_body_name`.
|
||||
|
||||
This key and its value will be present whenever the player comes close
|
||||
enough to a body for the Orbital Cruise/Glide HUD elements to appear.
|
||||
It will disappear again when they fly back above that altitude, or jump
|
||||
away.
|
||||
5. If Status.json does **not** have `BodyName` then clear `status_body_name`.
|
||||
6. For a `CodexEntry` event:
|
||||
1. Check that `status_body_name` is set. If it is not, exit.
|
||||
1. Set the EDDN `codexentry` schema message `BodyName` to this value.
|
||||
2. Check if it matches the `journal_body_name` value, and
|
||||
ONLY if they match, set `BodyID` in the EDDN `codexentry`
|
||||
schema message to the value of `journal_body_id`.
|
||||
|
||||
If `status_body_name` is not set then you MUST NOT include `BodyName` or
|
||||
`BodyID` keys/values in the EDDN message.
|
||||
|
||||
If `status_body_name` is set, but does not match with
|
||||
`journal_body_name` then you MUST NOT include a `BodyID` key+value in the
|
||||
EDDN message.
|
||||
|
||||
For emphasis, in both of these cases you MUST NOT include the keys with a
|
||||
`null`, `''`, or otherwise 'empty' value. Do not include the key(s) at all.
|
||||
|
||||
One possible issue is binary bodies where you might get an `ApproachBody` for
|
||||
one before descending towards the other, without an additional `ApproachBody`
|
||||
to correct things.
|
||||
|
||||
An example of this is `Baliscii 7 a` and `Baliscii 7 b`. Approaching one
|
||||
and going below Orbital Cruise altitude will set `journal_body_name` and
|
||||
`journal_body_id` to it, but you can then turn and approach the other
|
||||
without a new `ApproachBody` event, but `status_body_name` will change to
|
||||
the other when you are close enough.
|
||||
|
||||
In this case due to `status_body_name` and `journal_body_name` not matching
|
||||
the `codexentry` message MUST be sent **without** `BodyID`, but SHOULD be
|
||||
sent with the `status_body_name` value on the `BodyName` key.
|
||||
|
||||
e.g. for `Bestia A 2 a`
|
||||
```json
|
||||
"BodyName": "Bestia A 2 a",
|
||||
"BodyID": 15,
|
||||
```
|
||||
|
||||
If you cannot properly obtain the values for `BodyName` or `BodyID` then
|
||||
you MUST NOT include them.
|
||||
|
||||
## Receivers
|
||||
|
||||
As per ['BodyID and BodyName'](#bodyid-and-bodyname) above be aware that
|
||||
you are not guaranteed to receive these values for any given event. Some
|
||||
codex entries will be in space, and thus they aren't even relevant. In
|
||||
other cases it may not have been possible to properly determine both of them.
|
||||
|
||||
So you might receive any of:
|
||||
|
||||
1. Neither `BodyName` nor `BodyID` present in the message, not even the
|
||||
key names. This SHOULD indicate a codex entry object which is not on a
|
||||
body surface.
|
||||
2. `BodyName` key present with a value, but no `BodyID` key. This SHOULD
|
||||
indicate a codex entry object which is on a body surface, but probably
|
||||
where there is a close-orbiting binary companion which has confused things.
|
||||
3. Both `BodyName` and `BodyID` keys present, with values. This SHOULD
|
||||
indicate a codex entry object which is on a body surface.
|
||||
|
||||
Adjust your local processing accordingly.
|
||||
@ -1,6 +1,7 @@
|
||||
{
|
||||
"$schema" : "http://json-schema.org/draft-04/schema#",
|
||||
"id" : "https://eddn.edcd.io/schemas/codexentry/1#",
|
||||
"description" : "EDDN schema for CodexEntry Journal events. Full documentation at https://github.com/EDCD/EDDN/tree/master/schemas/codexentry-README.md",
|
||||
"type" : "object",
|
||||
"additionalProperties" : false,
|
||||
"required": [ "$schemaRef", "header", "message" ],
|
||||
@ -104,6 +105,12 @@
|
||||
"minLength" : 1
|
||||
}
|
||||
},
|
||||
"BodyID": {
|
||||
"type" : "integer"
|
||||
},
|
||||
"BodyName": {
|
||||
"type" : "string"
|
||||
},
|
||||
"IsNewEntry": {
|
||||
"$ref" : "#/definitions/disallowed",
|
||||
"description" : "Contains personal data"
|
||||
|
||||
@ -35,7 +35,7 @@
|
||||
"additionalProperties" : false,
|
||||
"required" : [ "timestamp", "event", "StarSystem", "StarPos", "SystemAddress", "BodyID" ],
|
||||
"properties" : {
|
||||
"timestamp":{
|
||||
"timestamp": {
|
||||
"type" : "string",
|
||||
"format" : "date-time"
|
||||
},
|
||||
|
||||
@ -13,8 +13,12 @@ class Validator(object):
|
||||
def addSchemaResource(self, schemaRef, schema):
|
||||
if schemaRef in self.schemas.keys():
|
||||
raise Exception("Attempted to redefine schema for " + schemaRef)
|
||||
schema = simplejson.loads(schema)
|
||||
self.schemas[schemaRef] = schema
|
||||
try:
|
||||
schema = simplejson.loads(schema)
|
||||
self.schemas[schemaRef] = schema
|
||||
|
||||
except simplejson.errors.JSONDecodeError as e:
|
||||
raise Exception('SCHEMA: Failed to load: ' + schemaRef)
|
||||
|
||||
def validate(self, json_object):
|
||||
results = ValidationResults()
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user