Product file restoration API

This page assumes you know how to authenticate your REST API requests

Restoration Principle

A product stored on REGARDS is either immediately downloadable, or stored on a nearline storage (like AWS Glacier), requiring a restoration action to become downloadable.

When a restoration is requested, there are several steps that are performed in the Catalog microservice:

• Products are retrieved from their URN through a request to ElasticSearch
• Products that are not accessible for the user are removed from the request
• The Catalog microservice calls the Storage microservice through its REST Client. Storage is the microservice responsible for making products available.

This guide introduce you 2 endpoints to make available nearline products in order to download product files synchronously :

• Check product availability endpoint gives you the status of the file, which indicates if the file can be immediately downloaded
• Product restoration endpoint makes nearline product files available to download

Usercase example

Here is an example of product file you need to restore in order be able to download it.
The product is not a reference (reference=false) and not online (online=false).

Some nearline product file - using standard lucene search engine

{
  "id": "URN:FEATURE:DATA:tenant:50fe35a3-e085-3eb1-bb61-86ab3fdeb7dd:V3",
  "providerId": "DATA-06-20220613-2390",
  "entityType": "DATA",
  ...,
  "files": {
    "RAWDATA": [
      {
        "dataType": "RAWDATA",
        "reference": false,
        "uri": "https://regards.host.com/api/v1/rs-catalog/downloads/URN:FEATURE:DATA:tenant:50fe35a3-e085-3eb1-bb61-86ab3fdeb7dd:V3/files/BF39BC048B52618838529D9D98043190",
        "mimeType": "application/xml",
        "online": false,
        "checksum": "BF39BC048B52618838529D9D98043190",
        "digestAlgorithm": "MD5",
        "filesize": 190,
        "filename": "iso.xml",
        "types": []
      },
      {
        ... 
      }
    ]
  }
}

Product URN

The value of the attribute id is an URN used to by Check product availability and Product restoration endpoints

Check product availability endpoint

There is one endpoint for check availability of single product and another endpoint for check availability of multiple products. Following example shows response of multiple product endpoint.

This endpoint checks for every file of product(s) if the file is available.

Here is the response format, with an example.

Array of objects (ProductFilesStatusDto) list of availability status of products

Check availability response: a file to restore

{
  "products": [
    {
      "files": [
        {
          "available": false,
          "checksum": "BF39BC048B52618838529D9D98043190",
          "expiration_date": "2020-12-31T00:00:00Z"
        }
      ],
      "id": "URN:FEATURE:DATA:tenant:50fe35a3-e085-3eb1-bb61-86ab3fdeb7dd:V3"
    }
  ]
}

The attribute available value can be:

• true - it means product file can be direct downloaded
• false - it means product file must be restored to be downloadable.

A file available can have an expiration_date, meaning that file will be not available after this date. It must be restored again to be downloadable again. Expiration date not indicated means REGARDS doesn't know when the file will be unavailable.

info

If some product file is missing from the response, it means you don't have access to product file, such product does not exist or some internal error occurred.
In that case, you have to contact REGARDS administrators.

Number of product limitations

In order to avoid flooding the storage, checking availability is limited to a certain number by API call, configurable in catalog microservice configuration.

Product restoration endpoint

Endpoint usage

Make product files with available=false restored. It has is no effect on file stored in an online storage or available=true.

There is one endpoint for single product restoration and another endpoint for multiple product restorations. These endpoints perform the action asynchronously, so they just return a code 200.
Once the restoration is done, a new call to the endpoint Check product availability endpoint will now mark the file as available (available=true)

info

By doing so, restoration ask storage microservice to make available these files.
After restoration, these files will be available for a configured time in hours.
Number of hours is configurable in catalog microservice.

Restored product files remains nearline

After restoration, the product file will still be displayed in the catalog search as offline (online=false).

Note that:

Logged user must have access right to this/these products

AMQP result

Once the file is restored, you can be notified about product restoration using an AMQP message

This message indicate if the restoration has been done successfully. If so, file is now available and ready to download.