third-party-mirrors/EDDN
1
0
Fork 0
mirror of https://github.com/EDCD/EDDN.git synced 2025-04-12 07:00:04 +03:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'beta'

Browse Source
This commit is contained in:
Athanasius 2022-06-01 12:52:03 +01:00
commit 12882b17da
No known key found for this signature in database
GPG Key ID: AE3E527847057C7D
13 changed files with 291 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

48
docs/Developers.md
View File

@ -301,6 +301,48 @@ Horizons-only features disabled.
active, but in the non-Odyssey game client case you only get the Horizons
boolean.
#### Other data Augmentations
Some schemas mandate that extra data be added, beyond what is in the source
data, to aid Listeners.
This is usually related to specifying which system an event took place in, and
usually means ensuring there is the full set of:
1. `StarSystem` - the name of the system.
2. `SystemAddress` - the game's unique numerical identifier for the system.
3. `StarPos` - The system's co-ordinates.
Whilst it can be argued that any Listener should see preceding event(s) that
give any missing information where at least the system name or `SystemAddress`
is already in the event data, this might not always be true. So Senders MUST
add this data where required. It helps to fill out basic system information
(name, SystemAddress and co-ordinates).
However, there is a known game bug that can result in it stopping writing to
the game journal, and some observed behaviour implies that it might then later
resume writing to that file, but with events missing. This means any Sender
that blindly assumes it knows the current system/location and uses that for
these Augmentations might send erroneous data.
1. **Senders MUST cross-check available event data with prior 'location'
event(s) to be sure the correct extra data is being added.**
2. **Listeners SHOULD realise that any data added as an Augmentation might be
in error.**
For Senders, if the source data only has `SystemAddress` then you MUST check
that it matches that from the prior `Location`, `FSDJump` or `CarrierJump`
event before adding `StarSystem` and `StarPos` data to a message. Drop the
message entirely if it does not match. Apply similar logic if it is only
the system's name that is already present in data. Do not blindly add
`SystemAddress` or `StarPos`. Likewise, do not blindly add `StarPos` if the
other data is already in the source, without cross-checking the system name
and `SystemAddress`.
Listeners might be able to apply their own cross-check on received messages,
and use any mismatch with respect to what they already know to make a decision
whether to trust the augmented data. Flagging it for manual review is probably
wise.
### Server responses
There are three possible sources of HTTP responses when sending an upload
to EDDN.
@ -435,7 +477,11 @@ data you will first need to zlib-decompress each message. Then you will
have a textual JSON object as per the Schemas.
In general, check the guidance for [Uploading messages](#uploading-messages)
for the expected format of the messages.
for the expected format of the messages. **Pay particular attention to any
schema-specific Augmentations**. Whilst Senders MUST make every effort to
ensure such data is correct it is possible that bugs in either their code, or
the game itself, could mean it is incorrect. Listeners use such data at
their own risk.
Consumers can utilise the `$schemaRef` value to determine which Schema a
particular message is for. There is no need to validate the messages

10
schemas/TEMPLATES/journalevent-README.md
View File

@ -106,10 +106,20 @@ value is what the name would have been in the source Journal data.
Please read [horizons and odyssey flags](../docs/Developers.md#horizons-and-odyssey-flags)
in the Developers' documentation.
#### StarSystem
You MUST add a `StarSystem` string containing the name of the system from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
## Listeners
The advice above for [Senders](#senders), combined with the actual Schema file
*should* provide all the information you need to process these events.

6
schemas/approachsettlement-README.md
View File

@ -64,6 +64,12 @@ You MUST add a StarSystem key/value pair representing the name of the system
this event occurred in. Source this from either Location, FSDJump or
CarrierJump as appropriate.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**

3
schemas/codexentry-README.md
View File

@ -27,6 +27,9 @@ in the Developers' documentation.
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
#### BodyID and BodyName
You SHOULD attempt to track the BodyName and BodyID where the player is
and add keys/values for these.

3
schemas/fssallbodiesfound-README.md
View File

@ -27,3 +27,6 @@ in the Developers' documentation.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**

49
schemas/fssbodysignals-README.md Normal file
View File

@ -0,0 +1,49 @@
# EDDN FSSAllBodiesFound Schema
## Introduction
Here we document how to take data from an ED `FSSBodySignals` Journal
Event and properly structure it for sending to EDDN.
Please consult [EDDN Schemas README](./README-EDDN-schemas.md) for general
documentation for a schema such as this.
If you find any discrepancies between what this document says and what is
defined in the relevant Schema file, then you should, in the first instance,
assume that it is the Schema file that is correct.
**PLEASE open
[an issue on GitHub](https://github.com/EDCD/EDDN/issues/new/choose)
to report any such anomalies you find so that we can check and resolve the
discrepancy.**
## Senders
The primary data source for this schema is the ED Journal event
`FSSBodySignals`.
### Augmentations
#### horizons and odyssey flags
Please read [horizons and odyssey flags](../docs/Developers.md#horizons-and-odyssey-flags)
in the Developers' documentation.
#### StarSystem
You MUST add a `StarSystem` string containing the name of the system from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
#### Remove _Localised key/values
All keys whose name ends with `_Localised`, i.e. the `Type_Localised`
key/values in Signals.
#### Examples:
```json
{ "timestamp":"2022-05-18T00:10:57Z", "event":"FSSBodySignals", "BodyName":"Phoi Auwsy ZY-Z d132 7 a", "BodyID":37, "SystemAddress":4546986398603, "Signals":[ { "Type":"$SAA_SignalType_Geological;", "Type_Localised":"Geological", "Count":2 } ] }
```

96
schemas/fssbodysignals-v1.0.json Normal file
View File

@ -0,0 +1,96 @@
{
"$schema" : "http://json-schema.org/draft-04/schema#",
"id" : "https://eddn.edcd.io/schemas/fssbodysignals/1#",
"type" : "object",
"additionalProperties" : false,
"required": [ "$schemaRef", "header", "message" ],
"properties": {
"$schemaRef": {
"type" : "string"
},
"header": {
"type" : "object",
"additionalProperties" : true,
"required" : [ "uploaderID", "softwareName", "softwareVersion" ],
"properties" : {
"uploaderID": {
"type" : "string"
},
"softwareName": {
"type" : "string"
},
"softwareVersion": {
"type" : "string"
},
"gatewayTimestamp": {
"type" : "string",
"format" : "date-time",
"description" : "Timestamp upon receipt at the gateway. If present, this property will be overwritten by the gateway; submitters are not intended to populate this property."
}
}
},
"message": {
"type" : "object",
"description" : "Contains all properties from the listed events in the client's journal, minus the Localised strings and the properties marked below as 'disallowed'",
"additionalProperties" : false,
"required" : [ "timestamp", "event", "StarSystem", "StarPos", "SystemAddress", "BodyID", "Signals" ],
"properties" : {
"timestamp": {
"type" : "string",
"format" : "date-time"
},
"event" : {
"enum" : [ "FSSBodySignals" ]
},
"horizons": {
"type" : "boolean",
"description" : "Whether the sending Cmdr has a Horizons pass."
},
"odyssey": {
"type" : "boolean",
"description" : "Whether the sending Cmdr has an Odyssey expansion."
},
"StarSystem": {
"type" : "string",
"minLength" : 1
},
"StarPos": {
"type" : "array",
"items" : { "type": "number" },
"minItems" : 3,
"maxItems" : 3,
"description" : "Must be added by the sender if not present in the journal event"
},
"SystemAddress": {
"type" : "integer"
},
"BodyID" : {
"type" : "integer"
},
"BodyName": {
"type" : "string",
"minLength" : 1
},
"Signals": {
"type" : "array",
"items" : {
"type" : "object",
"additionalProperties" : false,
"required" : [ "Type", "Count" ],
"properties" : {
"Type" : {
"type" : "string"
},
"Count" : {
"type" : "integer"
}
}
}
}
}
}
},
"definitions": {
"disallowed" : { "not" : { "type": [ "array", "boolean", "integer", "number", "null", "object", "string" ] } }
}
}

3
schemas/fssdiscoveryscan-README.md
View File

@ -27,3 +27,6 @@ in the Developers' documentation.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**

23
schemas/journal-README.md
View File

@ -72,15 +72,20 @@ You SHOULD add this key/value pair, using the value from the `LoadGame` event.
You SHOULD add this key/value pair, using the value from the `LoadGame` event.
#### StarSystem
You MUST add a `StarSystem` key/value pair representing the name of the
system this event occurred in. Source this from either `Location`,
`FSDJump` or `CarrierJump` as appropriate.
If not already present, you MUST add a `StarSystem` string containing the
name of the system from the last `FSDJump`, `CarrierJump`, or `Location` event.
#### SystemAddress
You MUST add a `SystemAddress` key/value pair representing the numerical ID
of the system this event occurred in. Source this from either `Location`,
`FSDJump` or `CarrierJump` as appropriate.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
This should only apply to `SAASignalsFound` events.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
If not already present, you MUST add a `StarPos` array containing the
system co-ordinates from the last `FSDJump`, `CarrierJump`, or `Location`
event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
This should only apply to `Docked`, `Scan` and `SAASignalsFound` events.

6
schemas/navbeaconscan-README.md
View File

@ -29,6 +29,12 @@ You MUST add a `StarSystem` key/value pair representing the name of the
system this event occurred in. Source this from either `Location`,
`FSDJump` or `CarrierJump` as appropriate.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**

3
schemas/scanbarycentre-README.md
View File

@ -30,3 +30,6 @@ in the Developers' documentation.
#### StarPos
You MUST add a `StarPos` array containing the system co-ordinates from the
last `FSDJump`, `CarrierJump`, or `Location` event.
**You MUST apply a location cross-check, as per
[Other data augmentations](../docs/Developers.md#other-data-augmentations).**

36
scripts/eddn-report-log-errors
View File

@ -62,14 +62,36 @@ def process_file(input_file: str) -> None:
# print(matches.group('sender_ip'))
# print('')
software_version = semantic_version.Version.coerce(matches.group('software_version'))
try:
software_version = semantic_version.Version.coerce(matches.group('software_version'))
except ValueError as e:
print(f"Error parsing sofwareVersion for:\n{matches.group('software_version')}\n{line}\n")
next
###################################################################
# Issues we know about and HAVE already alerted their
# developers to.
###################################################################
if matches.group('software_name') == 'EDDiscovery':
# https://github.com/EDDiscovery/EDDiscovery/releases/latest
if software_version >= semantic_version.Version.coerce('12.1.7.0'):
if software_version >= semantic_version.Version.coerce('15.0.0.0'):
if matches.group('schema_ref') == 'https://eddn.edcd.io/schemas/approachsettlement/1':
if matches.group('err_msg') == 'Failed Validation "[<ValidationError: "\'Latitude\' is a required property">]"':
# <https://github.com/EDDiscovery/EDDiscovery/issues/3236>
pass
elif matches.group('err_msg') == 'Failed Validation "[<ValidationError: "Additional properties are not allowed (\'Name_Localised\' was unexpected)">]"':
# <https://github.com/EDDiscovery/EDDiscovery/issues/3237>
pass
else:
print(line)
else:
print(line)
elif software_version >= semantic_version.Version.coerce('12.1.7.0'):
if matches.group('schema_ref') in (
'https://eddn.edcd.io/schemas/shipyard/2',
'https://eddn.edcd.io/schemas/outfitting/2',
@ -238,6 +260,16 @@ def process_file(input_file: str) -> None:
else:
print(line)
elif matches.group('schema_ref') == 'https://eddn.edcd.io/schemas/shipyard/2':
# <https://discord.com/channels/164411426939600896/205369618284544000/955030485791285258>
if matches.group('err_msg').startswith(
'Failed Validation "[<ValidationError: \'[] is too short\'>]"'
):
pass
else:
print(line)
else:
print(line)

31
src/eddn/conf/Settings.py
View File

@ -59,25 +59,28 @@ class _Settings(object):
"https://eddn.edcd.io/schemas/journal/1" : "schemas/journal-v1.0.json",
"https://eddn.edcd.io/schemas/journal/1/test" : "schemas/journal-v1.0.json",
"https://eddn.edcd.io/schemas/scanbarycentre/1" : "schemas/scanbarycentre-v1.0.json",
"https://eddn.edcd.io/schemas/scanbarycentre/1/test" : "schemas/scanbarycentre-v1.0.json",
"https://eddn.edcd.io/schemas/scanbarycentre/1" : "schemas/scanbarycentre-v1.0.json",
"https://eddn.edcd.io/schemas/scanbarycentre/1/test" : "schemas/scanbarycentre-v1.0.json",
"https://eddn.edcd.io/schemas/fssdiscoveryscan/1" : "schemas/fssdiscoveryscan-v1.0.json",
"https://eddn.edcd.io/schemas/fssdiscoveryscan/1/test" : "schemas/fssdiscoveryscan-v1.0.json",
"https://eddn.edcd.io/schemas/fssdiscoveryscan/1" : "schemas/fssdiscoveryscan-v1.0.json",
"https://eddn.edcd.io/schemas/fssdiscoveryscan/1/test" : "schemas/fssdiscoveryscan-v1.0.json",
"https://eddn.edcd.io/schemas/codexentry/1" : "schemas/codexentry-v1.0.json",
"https://eddn.edcd.io/schemas/codexentry/1/test" : "schemas/codexentry-v1.0.json",
"https://eddn.edcd.io/schemas/codexentry/1" : "schemas/codexentry-v1.0.json",
"https://eddn.edcd.io/schemas/codexentry/1/test" : "schemas/codexentry-v1.0.json",
"https://eddn.edcd.io/schemas/navbeaconscan/1" : "schemas/navbeaconscan-v1.0.json",
"https://eddn.edcd.io/schemas/navbeaconscan/1/test" : "schemas/navbeaconscan-v1.0.json",
"https://eddn.edcd.io/schemas/navbeaconscan/1" : "schemas/navbeaconscan-v1.0.json",
"https://eddn.edcd.io/schemas/navbeaconscan/1/test" : "schemas/navbeaconscan-v1.0.json",
"https://eddn.edcd.io/schemas/navroute/1" : "schemas/navroute-v1.0.json",
"https://eddn.edcd.io/schemas/navroute/1/test" : "schemas/navroute-v1.0.json",
"https://eddn.edcd.io/schemas/navroute/1" : "schemas/navroute-v1.0.json",
"https://eddn.edcd.io/schemas/navroute/1/test" : "schemas/navroute-v1.0.json",
"https://eddn.edcd.io/schemas/approachsettlement/1" : "schemas/approachsettlement-v1.0.json",
"https://eddn.edcd.io/schemas/approachsettlement/1/test" : "schemas/approachsettlement-v1.0.json",
"https://eddn.edcd.io/schemas/fssallbodiesfound/1" : "schemas/fssallbodiesfound-v1.0.json",
"https://eddn.edcd.io/schemas/fssallbodiesfound/1/test" : "schemas/fssallbodiesfound-v1.0.json",
"https://eddn.edcd.io/schemas/approachsettlement/1" : "schemas/approachsettlement-v1.0.json",
"https://eddn.edcd.io/schemas/approachsettlement/1/test" : "schemas/approachsettlement-v1.0.json",
"https://eddn.edcd.io/schemas/fssallbodiesfound/1" : "schemas/fssallbodiesfound-v1.0.json",
"https://eddn.edcd.io/schemas/fssallbodiesfound/1/test" : "schemas/fssallbodiesfound-v1.0.json",
"https://eddn.edcd.io/schemas/fssbodysignals/1" : "schemas/fssbodysignals-v1.0.json",
"https://eddn.edcd.io/schemas/fssbodysignals/1/test" : "schemas/fssbodysignals-v1.0.json",
}
GATEWAY_OUTDATED_SCHEMAS = [