Skip to main content
Version: 1.14.0

REGARDS OAIS implementation

OAIS recommendation

REGARDS allows to populate its catalog with products by following the OAIS recommendation. You can send your SIP (Submission Information Package) using the web HMI, the REST API or directly throught the bus message of the Ingest microservice.

Overview

To do so, administrator can provide a Json file containing :

  • metadata information about how to handle features to acquire
  • features list of SIP to acquire, with geographical features and metadata

Exemple :

{
"type": "FeatureCollection",
"metadata": {
"sessionOwner": "ISS Mission #1",
"session": "Monday acquisition",
"ingestChain": "DefaultProcessingChain",
"categories": ["ISS_MISSION1_DATA"],
"versioningMode": "INC_VERSION",
"replaceErrors": false,
"storages": [
{
"pluginBusinessId": "Local",
"storePath" : "/demo/exemple-1",
"targetTypes": []
}
]
},
"features": [
{
"type": "Feature",
"id": "regards-2296-data-1",
"ipType": "DATA",
"geometry": null,
"properties": {
"contentInformations": [
{
"dataObject": {
"regardsDataType": "RAWDATA",
"filename": "data-1.dat",
"locations": [
{
"url": "file:///regards/demo/data-1.dat"
}
],
"checksum": "9a964ed3be0e2e2786d82ace9d971e90",
"algorithm": "MD5"
},
"representationInformation": {
"syntax": {
"name": "TEXT",
"mimeType": "text/plain"
}
}
}
],
"pdi": {
"contextInformation": {
"tags": ["KEYWORD"]
},
"provenanceInformation": { "history": [] },
"accessRightInformation": {}
},
"descriptiveInformation": {
"longValue": 11056,
"creationdate": "2018-01-20T17:22:48Z",
"doubleValue": 35.56598,
"dataSetType": "type_01",
"label": "REGARDS 2296 generated data -1"
}
}
}
]
}

FeatureCollection structure

Metadata

  "metadata": {
"sessionOwner": "ISS Mission #1",
"session": "Monday acquisition",
"ingestChain": "DefaultProcessingChain",
"categories": ["ISS_MISSION1_DATA"],
"versioningMode": "INC_VERSION",
"replaceErrors": false,
"storages": [
{
"pluginBusinessId": "Local",
"storePath" : "/demo/exemple-1",
"targetTypes": [],
"size": {
"min": 10,
"max" : 10000,
}
}
]
}
PropertyMandatoryDescriptionPossible values
sessionOwnerxIdentifier of the data provider. Used to monitor all products of a same providerany text, max 128 characters
sessionxIdentifier of the current group of data. Used to monitor a subset of the session products.any text, max 128 characters
ingestChainxName of the configured ingest chain. This chain can be configured through administration HMI and contains all the configuration to generate the final product, the AIP, from the provided SIP.Any chain name configured
categoriesList of keywords. Thoses keywords can be used lately to select products to add in the metacatalogany text
versioningModexHow to handle new version of same product (same feature id)IGNORE, INC_VERSION, MANUAL, REPLACE
replaceErrorsTrue to delete previous requests in error state with the same feature id. Default value is falsetrue, false(default)
storagesxList of storage configuration that define how to store physical files associated to the provided list of features-

Versionning mode values :

  • IGNORE : New version of same feature are not ingested.
  • INC_VERSION : New version of same feature are ingested with version incremented.
  • MANUAL : Wait for project administrator to choose how to handle new version.
  • REPLACE : New version of same feature are ingested with version incremented and previous versions are deleted.

storages

PropertyMandatoryDescriptionPossible values
pluginBusinessIdxName of the configured storage. Each storage have to be configured though the administrator HMIAny configured storage label
storePathOptional property to define a directory into the storage location to store each filessubdirectory path
targetTypesOptional property to define which REGARDS type of files need to be stored in the current storage. If empty all files are stored in the storageRAWDATA , QUICKLOOK_SD, QUICKLOOK_MD, QUICKLOOK_HD, DOCUMENT, THUMBNAIL, AIP, DESCRIPTION
sizeOptional property to define accepted file sizes on this storage location. At least one property among min and max is mandatory to limit the size, otherwise all sizes will be accepted.

Features

{
"type": "Feature",
"id": "regards-2296-data-1",
"ipType": "DATA",
"geometry": {
"crs" : "WGS84",
"type" : "Point",
"coordinates": [30, 10]
},
"properties": {
"contentInformations": [
{
"dataObject": {
"regardsDataType": "RAWDATA",
"filename": "regards-2296-data-1.dat",
"locations": [
{
"url": "file:///regards-input/validation/data/2296/regards-2296-data-1.dat"
}
],
"checksum": "9a964ed3be0e2e2786d82ace9d971e90",
"algorithm": "MD5"
},
"representationInformation": {
"syntax": {
"name": "TEXT",
"mimeType": "text/plain"
}
}
}
],
"pdi": {
"contextInformation": {
"tags": ["KEYWORD"]
},
"provenanceInformation": { "history": [] },
"accessRightInformation": {}
},
"descriptiveInformation": {
"property1": 11056,
"date": "2018-01-20T17:22:48Z",
"property2": 35.56598,
"property3": ["type_01", "type_02"],
"property4": "REGARDS demo",
"property5": [ 10, 20 ],
}
}
}

A feature is a product to acquire in REGARDS.

Feature

PropertyMandatoryDescriptionPossible values
typexFeature type. Fixed valueFeature
idx(Your) Identifier of the product to acquire. This identifier must be unique. If two features have the same id then we consider there is two different versions of the same productany text
ipTypexREGARDS type of featureCOLLECTION, DATA, DATASET
geometryStandard GeoJson geometry of the featureRFC7946 Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon
contentInformationsxFeature associated filesCan be empty
pdixPreservation Description Information-
descriptiveInformationxA group key/value containing all specific information about the product-

ContentInformation

dataObject

"dataObject": {
"regardsDataType": "RAWDATA",
"filename": "regards-2296-data-1.dat",
"locations": [
{
"url": "file:///regards-input/validation/data/2296/regards-2296-data-1.dat"
}
],
"checksum": "9a964ed3be0e2e2786d82ace9d971e90",
"algorithm": "MD5"
},
PropertyMandatoryDescriptionPossible values
regardsDataTypexREGARDS type of fileRAWDATA , QUICKLOOK_SD, QUICKLOOK_MD, QUICKLOOK_HD, DOCUMENT, THUMBNAIL, AIP, DESCRIPTION
filenamexName of the file to acquire-
locationsxLocation of the file to acquire. Locations are described under this table.-
checksumxChecksum of the file to acquire-
algorithmxAlgorithm used to generate file checksumMD5...
fileSizeSize in bytes of the file to acquire. This property will be automaticaly calculated if the file to acquire is directly accessible-

dataObject locations

"locations": [
{
"url": "file:///regards/demo/data-1.dat"
},
{
"storage": "FTP Server",
"url": "ftp:/172.125.10.1/regards/demo/ftp/data-1.dat"
},
],
PropertyMandatoryDescriptionPossible values
storageStorage location of the file. If this property is provided, then the file is already stored and REGARDS will only keep the reference to the file. There will be no move file operation. If the storage is a configured storage in REGARDS system so the file can be accessed, if not the file will be considered as offline.Any text
urlxLocation of the file. If previous property storage is not provided, then the file will be copyed from this url to the storages defined in the metadata bloc, so ensure this url must be an accessible file:/// url protocol from the Storage microservice. If a storage has been provided then this url can be any text that can be a known url for the given storage.any text

NOTE : To reference an online file accessible through HTTP protocol, you have have to use the reference system as the exemple bellow. With this reference the file will be known by REGARDS as an online referenced file and will be accessible without any needs to configure a storage location associated. With this type of reference, the file url provided by regards catalog will be the original provided url.

"locations": [
{
"storage": "web",
"url": "https://regardsoss.github.io/appendices/oais"
}
],

representationInformation

"representationInformation": {
"semantic": {},
"syntax": {
"name": "TEXT",
"mimeType": "text/plain"
}
}
PropertyMandatoryDescriptionPossible values
semantic-
namexName of the mime type of the file
mimeTypexMime type of the file to acquire
heightOptional property to define in pixel the height of an image. This property will be automaticaly fill if the file to acquire is an image and is directly accessibleNumber in pixel
widthOptional property to define in pixel the height of an image. This property will be automaticaly fill if the file to acquire is an image and is directly accessibleNumber in pixel

Preservation Description Information (pdi)

"pdi": {
"contextInformation": {
"tags": ["KEYWORD"]
},
"provenanceInformation": { "history": [] },
"accessRightInformation": {}
},
PropertyMandatoryDescriptionPossible values
tagskeywords to add to each featureany text
provenanceInformationDescription of the origine and history of the feature
accessRightInformationDescription of the feature access rights

REGARDS Data types

TypeDescription
RAWDATAType to reference a data file.
QUICKLOOK_SDType to reference a small resolution image of the product preview
QUICKLOOK_MDType to reference a medium resolution image of the product preview
QUICKLOOK_HDType to reference a large resolution image of the product preview
DOCUMENTType to reference a document associated to the product
THUMBNAILType to reference a thumbnail of the product
AIPType to reference the matadata file of the product
DESCRIPTIONType to reference a description of the product