third-party-mirrors/EDDN
1
0
Fork 0
mirror of https://github.com/EDCD/EDDN.git synced 2025-04-23 03:50:30 +03:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #151 from EDCD/enhancement/document-all-schemas

Browse Source 
Document all of the schemas in their own README
This commit is contained in:
Athanasius 2021-11-04 11:26:04 +00:00 committed by GitHub
commit ac0a872d30
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 491 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

79
schemas/README-EDDN-schemas.md
View File

@ -62,13 +62,27 @@ the available endpoints and how they work. There is some
[third-party documentation](https://github.com/Athanasius/fd-api/blob/main/docs/README.md)
by Athanasius.
It is *not* recommended to use CAPI data as the source as it's fraught with
additional issues. EDMarketConnector does so in order to facilitate
obtaining data without the player needing to open the commodities screen.
#### Detecting CAPI data lag
When using the Companion API please be aware that the server that supplies this
data sometimes lags behind the game - usually by a few seconds, sometimes by
minutes. You MUST check in the data from the CAPI that the Cmdr is
docked (`["commander"]["docked"]` is `True`) and that the station and
system (`["lastStarport"]["name"]` and `["lastSystem"]["name"]`) match those
docked, and that the station and system names match those
reported from the Journal before using the data for the commodity, outfitting
and shipyard schemas.
and shipyard schemas:
1. Retrieve the commander data from the `/profile` CAPI endpoint.
2. Check that `commander['docked']` is true. If not, abort.
3. Retrieve the data from the `/market` and `/shipyard` CAPI endpoints.
4. Compare the system and station name from the CAPI market data,
`["lastStarport"]["name"]` and `["lastSystem"]["name"]`,
to that from the last `Docked` or `Location` journal event. If either does
not match then you MUST **abort**. This likely indicates that the CAPI
data is lagging behind the game client state and thus should not be used.
---
@ -170,39 +184,46 @@ Each `message` object must have, at bare minimum:
much more than this. Again, consult the
[schemas and their documentation](./).
Note that many of the key names chosen in the schemas are based on the CAPI
data, not Journal events, because the CAPI came first. This means renaming
many of the keys from Journal events to match the schema.
EDDN is intended to transport generic data not specific to any particular Cmdr
and to reflect the data that a player would see in-game in station services or
the local map. To that end, uploading applications MUST ensure that messages do
not contain any Cmdr-specific data (other than "uploaderID" and the "horizons"
flag). In practice as of E:D 3.3 this means:
flag).
* commodity: Skip commodities with `"categoryname": "NonMarketable"` (i.e.
Limpets - not purchasable in station market) or `"legality":` *non-empty
string* (not normally traded at this station market).
* outfitting: Skip items whose availability depends on the Cmdr's status rather
than on the station. Namely:
- Items that aren't weapons/utilities (`Hpt_*`), standard/internal
modules (`Int_*`) or armour (`*_Armour_*`) (i.e. bobbleheads, decals,
paintjobs and shipkits).
- Items that have a non-null `"sku"` property, unless
it's `"ELITE_HORIZONS_V_PLANETARY_LANDINGS"` (i.e. PowerPlay and tech
broker items).
- The `"Int_PlanetApproachSuite"` module (for historical reasons).
* shipyard: *Include* ships listed in the `"unavailable_list"` property (i.e.
available at this station, but not to this Cmdr).
* journal: Strip out `"..._Localised"` properties throughout the data
structure.
* journal/Docked: Strip out `"Wanted"`, `"ActiveFine"`, `"CockpitBreach"`
properties
* journal/FSDJump: Strip out `"Wanted"`, `"BoostUsed"`, `"FuelLevel"`
, `"FuelUsed"` and `"JumpDist"` properties.
* journal/Location: Strip out `"Wanted"`, `"Latitude"` and `"Longitude"`
properties.
* journal/Location and journal/FSDJump: strip out `"HappiestSystem"`
, `"HomeSystem"`, `"MyReputation"` and `"SquadronFaction"` properties within
the list of `"Factions"`.
The individual schemas will instruct you on various elisions (removals) to
be made to comply with this.
Some of these requirements are also enforced by the schemas, and some things
the schemas enforce might not be explicitly called out here, so **do**
check what you're sending against the schema when implementing sending new
events.
## Receiving messages
EDDN provides a continuous stream of information from uploaders. To use this
data you'll need to connect to the stream using ZeroMQ (a library is probably
available for your language of choice).
The URL for the live Relay is:
tcp://eddn.edcd.io:9500
Once you've connected to that you will receive messages. To access the
data you will first need to zlib-decompress each message. Then you will
have a textual JSON object as per the schemas.
In general, check the guidance for [Uploading messages](#uploading-messages)
for the expected format of the messages.
Consumers can utilise the `$schemaRef` value to determine which schema a
particular message is for. There is no need to validate the messages
against the schemas yourself, as that is performed on the EDDN Gateway.
Messages that do not pass the schema validation there are not forwarded to
receivers.
There is [example code](https://github.com/EDCD/EDDN/tree/master/examples)
available for a variety of programming languages to help you get started.

55
schemas/blackmarket-README.md Normal file
View File

@ -0,0 +1,55 @@
# EDDN BlackMarket Schema
## Introduction
**This schema is deprecated. The same data has been made available via the
`prohibited` array in the [commodity schema](./commodity-v3.0.json) since
September 2017.**
What follows below is only for historical curiosity.
---
Here we document how to take data from an ED `MarketSell` Journal Event and
properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event `MarketSell`.
### Key Renames
#### name
Due to how the EDDN schema is defined the `Type` key/value should
have the key renamed to `name`.
#### prohibited
Due to how the EDDN schema is defined the `IllegalGoods` key/value should
have the key renamed to `prohibited`.
#### marketID
The Journal documentation says this is `MarketID`, but in the schema the
`m` is lower case.
### Elisions
You MUST remove the following key/value pairs from the data:
- `TotalSale`
- `AvgPricePaid`
- `StolenGoods`
- `BlackMarket` - Because we're using this schema, so this is un-necessary.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### systemName
The star system name for where this market is. Use the `StarSystem` value
from the prior `Docked` or `Location` event.
#### stationName
From the `StationName` value on the prior `Docked` or `Location` event.

6
schemas/codexentry-README.md
View File

@ -17,6 +17,12 @@ You MUST remove any key where the key name ends in
You MUST remove the two keys `IsNewEntry` and `NewTraitsDiscovered`.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### StarPos
You MUST **add** a `StarPos` key with value of type `array` containing the
galaxy co-ordinates of the system. You will need to have obtained these

94
schemas/commodity-README.md Normal file
View File

@ -0,0 +1,94 @@
# EDDN Commodity Schema
## Introduction
Here we document how to take data from an ED `Market` Journal Event and
properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event `Market`,
and the additional file, `Market.json`, that it signals the writing of.
So, look for the `Market` event, and when it occurs open and read the
`Market.json` file which contains the actual data. Treat *that* data as
the event.
It *is* also possible to construct this data from a CAPI `/market` query.
See [Using CAPI data](#using-capi-data) below.
### Statusflags
`statusFlags` is an optional augmentation that can be used to identify
`Producer`, `Consumer`, and `Rare` goods from the output of `Market.json`.
So, as per the schema, do include it if available.
### Key Renames
Many of the key names have a different case defined in this schema, make
sure you are renaming them as appropriate.
#### StarSystem to systemName
Rename the `StarSystem` key name to `systemName`.
### Elisions
#### Remove _Localised key/values
All keys whose name ends with `_Localised`, i.e. the `Name_Localised`
key/values in Items.
#### Other Elisions
You MUST remove the following key/value pairs from the data:
- `StationType` key/value.
- `Producer` key/value pair in Items.
- `Rare` key/value pair in Items.
- `id` key/value pair in Items.
In the list of commodites:
- Skip commodities with `"categoryname": "NonMarketable"` (i.e.
Limpets - not purchasable in station market) or a *non-empty*`"legality":`
string (not normally traded at this station market).
#### Item Category
Remove not only the `Category_Localised` key/value, as above, but also the
`Category` key/value pair from each Item.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
### Using CAPI data
It is *not* recommended to use CAPI data as the source as it's fraught with
additional issues. EDMarketConnector does so in order to facilitate
obtaining data without the player needing to open the commodities screen.
Please read
[the guidance on checking for CAPI lag](README-EDDN-schemas.md#detecting-capi-data-lag)
before utilising CAPI data for EDDN messages.
Note that CAPI `/market` data will sometimes have the `StatusFlasg` per
item, which are defined as optional in this schema (because they're not in
the Market.json data). You SHOULD include this data in your message if
using CAPI as the source.
Now you will need to construct the necessary additional fields:
#### CAPI horizons flag
If your application can be certain that the game client is still running,
and logged into the game (not just run to the main menu), then you can
simply use the value from the `LoadGame` journal event.
Otherwise, you MUST check if any of the economies from the `/market`
data have a `name` of `Colony`, if so, set this flag true.
Additionally, you should retrieve the CAPI `/shipyard` endpoint and check if
any of the listed ships or modules have a `sku` value of
`ELITE_HORIZONS_V_PLANETARY_LANDINGS`. If so, set this flag true.
#### CAPI odyssey flag
Unfortunately there is no method to be *certain* of this from CAPI data, so
you will have to trust in the system/station name check and use the value
from the Journal `LoadGame` event.

32
schemas/fssdiscoveryscan-README.md Normal file
View File

@ -0,0 +1,32 @@
# EDDN FSSDiscoveryScan Schema
## Introduction
Here we document how to take data from an ED `FSSDiscoveryScan` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event
`FSSDiscoveryScan`.
### Key Renames
Many of the key names have a different case defined in this schema, make
sure you are renaming them as appropriate.
### Elisions
You MUST remove the following key/value pairs from the data:
- `Progress` key/value pair.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.

86
schemas/journal-README.md Normal file
View File

@ -0,0 +1,86 @@
# EDDN Journal Schema
## Introduction
Here we document how to take data from miscellaneous ED Journal
events and properly structure it for sending to EDDN.
This is the historical "all Journal events" schema that will be deprecated
in the future. Please check for a schema specific to the journal event
under consideration to see if data should be sent on that event specific
schema instead.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal events:
- `Docked`
- `FSDJump`
- `Scan`
- `Location`
- `SAASignalsFound`
- `CarrierJump`
- `CodexEntry` - But see the separate
[codexentry schema](./codexentry-README.md) documentation.
### Key Renames
Many of the key names have a different case defined in this schema, make
sure you are renaming them as appropriate.
### Elisions
#### Remove _Localised key/values
All keys whose name ends with `_Localised`, i.e. the `Name_Localised`
key/values in Items.
#### Personal data in `Docked` events
The following keys+values should be removed from `Docked` event data:
- `Wanted`
- `ActiveFine`
- `CockpitBreach`
#### Personal data in `FSDJump` events
The following keys+values should be removed from `FSDJump` event data:
- `Wanted`
- `BoostUsed`
- `FuelLevel`
- `FuelUsed`
- `JumpDist`
- `HappiestSystem` from within the list of `Factions`.
- `HomeSystem` from within the list of `Factions`.
- `MyReputation` from within the list of `Factions`.
- `SquadronFaction` from within the list of `Factions`.
#### Personal data in `Location` events
The following keys+values should be removed from `Location` event data:
- `Wanted`
- `Latitude`
- `Longitude`
- `HappiestSystem` from within the list of `Factions`.
- `HomeSystem` from within the list of `Factions`.
- `MyReputation` from within the list of `Factions`.
- `SquadronFaction` from within the list of `Factions`.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### StarSystem
You MUST add a `StarSystem` key/value pair representing the name of the
system this event occurred in. Source this from either `Location`,
`FSDJump` or `CarrierJump` as appropriate.
#### SystemAddress
You MUST add a `SystemAddress` key/value pair representing the numerical ID
of the system this event occurred in. Source this from either `Location`,
`FSDJump` or `CarrierJump` as appropriate.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.

31
schemas/navbeaconscan-README.md Normal file
View File

@ -0,0 +1,31 @@
# EDDN NavBeaconScan Schema
## Introduction
Here we document how to take data from an ED `NavBeaconScan` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event
`NavBeaconScan`.
### Elisions
There are no elisions in this schema.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### StarSystem
You MUST add a `StarSystem` key/value pair representing the name of the
system this event occurred in. Source this from either `Location`,
`FSDJump` or `CarrierJump` as appropriate.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.

29
schemas/navroute-README.md Normal file
View File

@ -0,0 +1,29 @@
# EDDN NavRoute Schema
## Introduction
Here we document how to take data from an ED `NavRoute` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the `NavRoute.json` file. That
it has been freshly written is signalled by the ED Journal event `NavRoute`.
So, monitor the Journal as normal, and when you see a `NavRoute` event open
the `NavRoute.json` file for reading, read it, and close it again. Use the
data you got from reading this file, not merely the Journal event.
The primary data to be sent is the `Route` array from the contents of the
separate file.
### Elisions
There are no elisions in this schema.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.

42
schemas/outfitting-README.md Normal file
View File

@ -0,0 +1,42 @@
# EDDN Outfitting Schema
## Introduction
Here we document how to take data from an ED `Outfitting` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event
`Outfitting`.
You MAY also source this data from the CAPI `/shipyard` endpoint.
Please read
[the guidance on checking for CAPI lag](README-EDDN-schemas.md#detecting-capi-data-lag)
before utilising CAPI data for EDDN messages.
You only need the `name` key's value for each member of the `modules` array.
### Key Renames
Many of the key names have a different case defined in this schema, make
sure you are renaming them as appropriate.
### Elisions
Remove items whose availability depends on the Cmdr's status rather than on the
station. Namely:
- Items that aren't weapons/utilities (`Hpt_*`), standard/internal
modules (`Int_*`) or armour (`*_Armour_*`) (i.e. bobbleheads, decals,
paintjobs and shipkits). This is enforced by the schema.
- Items that have a non-null `"sku"` property, unless
it's `"ELITE_HORIZONS_V_PLANETARY_LANDINGS"` (i.e. PowerPlay and tech
broker items).
- The `"Int_PlanetApproachSuite"` module (for historical reasons).
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.

26
schemas/scanbarycentre-README.md Normal file
View File

@ -0,0 +1,26 @@
# EDDN ScanBaryCentre Schema
## Introduction
Here we document how to take data from an ED `ScanBaryCentre` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event
`ScanBaryCentre`.
### Elisions
There are no elisions in this schema.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.

40
schemas/shipyard-README.md Normal file
View File

@ -0,0 +1,40 @@
# EDDN Shipyard Schema
## Introduction
Here we document how to take data from an ED `Shipyard` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
## Senders
The primary data source for this schema is the ED Journal event
`Shipyard`.
You MAY also source this data from the CAPI `/shipyard` endpoint.
Please read
[the guidance on checking for CAPI lag](README-EDDN-schemas.md#detecting-capi-data-lag)
before utilising CAPI data for EDDN messages.
You only need the `name` key's value for each member of the `PriceList`
array (if using Journal, it will be from the `ships` array if using CAPI
data).
When using CAPI data *include* ships listed in the `"unavailable_list"`
property (i.e. available at this station, but not to this Cmdr).
This list of ship names will go in the `ships` array in the EDDN message.
### Key Renames
Many of the key names have a different case defined in this schema, make
sure you are renaming them as appropriate.
### Elisions
There are no elisions in this schema.
### Augmentations
#### horizons flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### odyssey flag
You SHOULD add this key/value pair, using the value from the `LoadGame` event.