V1.15 to V2.0

Steps to migrate REGARDS from version 1.15 to 2.0

Playbook modifications

• Download last playbook version and move your inventory inside the new playbook
• Edit your inventory file group_vars/all/main.yml :

# Replace 
version: 1.15.X
# Into 
version: 2.0.0

• Move static files from group_vars/regards_nodes/files to static:

• Move static files from group_vars/regards_nodes/files to static:

cd path/to/your/inventory/group_vars/regards_nodes/
mv ./files ../../static

• Edit your inventory file group_vars/regards_nodes/main.yml and make these changes:

# 1- Set property group_docker_static_files_migrated to True
# Before
group_docker_static_files_migrated: False
# After
group_docker_static_files_migrated: True

# 2- Add new property legacy_spring to False
group_config_mservices:
  legacy_spring: False
  [...]

• Ensure all plugins, inside your inventory file group_vars/regards_nodes/main.yml, end with -plugin postfix, like this:

# Before
group_docker_plugins:
  [...]
  - image: rs-worker-manager-sender
    tag: "{{ group_docker_tags.plugins }}"
  [...]
# After
group_docker_plugins:
  [...]
  - image: rs-worker-manager-sender-plugin
    tag: "{{ group_docker_tags.plugins }}"
  [...]

• Remove the manual_revision property on RabbitMQ service:

# Before
group_docker_cots:
  [...]
  rabbitmq:
    tag: "{{ group_docker_tags.cots }}"
    manual_revision: xxx

# After
group_docker_cots:
  [...]
  rabbitmq:
    tag: "{{ group_docker_tags.cots }}"
    [...]

• Shutdown REGARDS using the playbook

ansible-playbook -i [...] regards-shutdown.yml [...]

• Remove old plugins from NFS as their name may have changed

ansible-playbook -i [...] regards-delete-plugins.yml [...]

• Update REGARDS using the playbook

ansible-playbook -i [...] regards.yml [...]

During startup and before RabbitMQ migration, some REGARDS services will be unavailable.

RabbitMQ exchange and queues migration

• Wait RabbitMQ to be up and running

cd /path/to/regards/cli
./status.sh
# Check every containers of RabbitMQ are up and running

• Now, connect to your RabbitMQ node or one of its replicate

# Enter inside one of the RabbitMQ container
./exec.sh rs-rabbitmq bash

• Create a migration script using vi /var/lib/rabbitmq/mnesia/migration-to-2.0-backup-messages.sh with following content:

#!/bin/bash -e

# Param 1 : Shovel ID
# Param 2 : Source tag
# Param 2 : Dest tag
function add_shovel
{
  local shovelId=${1}
  local srcQueue=${2}
  local destQueue=${3}

  printf >&2 "[\033[32mSHOVEL\033[m]\tAdd shovel to move ${srcQueue} to ${destQueue}\n"
  rabbitmqctl --quiet set_parameter shovel migrate-${shovelId} \
  "{\"src-protocol\": \"amqp091\", \"src-uri\": \"amqp:///regards.multitenant.manager\", \"src-queue\": \"${srcQueue}\", \"dest-protocol\": \"amqp091\", \"dest-uri\": \"amqp:///regards.multitenant.manager\", \"dest-queue\": \"${destQueue}\"}"
}

# Param 1 : Shovel ID
function delete_shovel
{
  local shovelId=${1}

  printf >&2 "[\033[32mSHOVEL\033[m]\tRemove shovel ${shovelId}\n"
  rabbitmqctl --quiet clear_parameter shovel migrate-${shovelId}
}


# Param 1 : Queue Name to delete
# Ensure queue is empty
function delete_empty_queue
{
  local queueName=${1}

  printf >&2 "[\033[32mQUEUES\033[m]\tRemove queue ${queueName}\n"
  rabbitmqctl --quiet delete_queue --vhost=regards.multitenant.manager ${queueName} --if-unused --if-empty || printf >&2 "[\033[34mQUEUES\033[m]\tQueue ${queueName} not existing, ignoring error...\n"
}


# Param 1 : Queue Name to delete
# Do not check if queue is empty
function delete_queue
{
  local queueName=${1}

  printf >&2 "[\033[32mQUEUES\033[m]\tRemove queue ${queueName}\n"
  rabbitmqctl --quiet delete_queue --vhost=regards.multitenant.manager ${queueName} --if-unused || printf >&2 "[\033[34mQUEUES\033[m]\tQueue ${queueName} not existing, ignoring error...\n"
}

# Param 1 : Exchange Name to delete
function delete_exchange
{
  local exchangeName=${1}

  printf >&2 "[\033[32mEXCHANGES\033[m]\tRemove exchange ${exchangeName}\n"
  rabbitmqadmin -V regards.multitenant.manager delete exchange name="${exchangeName}" || printf >&2 "[\033[34mEXCHANGES\033[m]\tExchange ${exchangeName} not existing, ignoring error...\n"
}

# Param 1 : Queue Name to create
function create_tmp_queue
{
  local queueName=${1}
  printf >&2 "[\033[32mQUEUES\033[m]\tCreate queue ${queueName}\n"
  rabbitmqadmin declare queue --vhost=regards.multitenant.manager name=${queueName} durable=true
}

create_tmp_queue tmp.WorkerManagerResponse
create_tmp_queue tmp.NotificationRequestEventHandler
create_tmp_queue tmp.SpecificRecipientNotificationRequestEventHandler

add_shovel 01 \
   regards.worker.manager.response \
   tmp.WorkerManagerResponse

add_shovel 02 \
  regards.broadcast.rs-notifier.fr.cnes.regards.modules.notifier.service.flow.NotificationRequestEventHandler \
  tmp.NotificationRequestEventHandler

add_shovel 03 \
  regards.broadcast.rs-notifier.fr.cnes.regards.modules.notifier.service.flow.SpecificRecipientNotificationRequestEventHandler \
  tmp.SpecificRecipientNotificationRequestEventHandler

printf >&2 "[\033[32mSHOVEL\033[m]\tWait 30s to ensure messages have been moved to tmp queues\n"
sleep 30

delete_shovel 01
delete_shovel 02
delete_shovel 03

delete_empty_queue regards.worker.manager.response
delete_empty_queue regards.broadcast.rs-notifier.fr.cnes.regards.modules.notifier.service.flow.NotificationRequestEventHandler
delete_empty_queue regards.broadcast.rs-notifier.fr.cnes.regards.modules.notifier.service.flow.SpecificRecipientNotificationRequestEventHandler

• Execute the script:

chmod +x /var/lib/rabbitmq/mnesia/migration-to-2.0-backup-messages.sh
/var/lib/rabbitmq/mnesia/migration-to-2.0-backup-messages.sh
rm -f /var/lib/rabbitmq/mnesia/migration-to-2.0-backup-messages.sh

• Get out of the RabbitMQ container, wait 5 min, and ensure REGARDS is running fine (using status.sh)

• Now you can start the final RabbitMQ migration step. Connect to your RabbitMQ node or one of its replicate

# Enter inside one of the RabbitMQ container
./exec.sh rs-rabbitmq bash

• Create a migration script using vi /var/lib/rabbitmq/mnesia/migration-to-2.0-restore-messages.sh with following content:

#!/bin/bash -e

# Param 1 : Shovel ID
# Param 2 : Source tag
# Param 2 : Dest tag
function add_shovel
{
  local shovelId=${1}
  local srcQueue=${2}
  local destQueue=${3}

  printf >&2 "[\033[32mSHOVEL\033[m]\tAdd shovel to move ${srcQueue} to ${destQueue}\n"
  rabbitmqctl --quiet set_parameter shovel migrate-${shovelId} \
  "{\"src-protocol\": \"amqp091\", \"src-uri\": \"amqp:///regards.multitenant.manager\", \"src-queue\": \"${srcQueue}\", \"dest-protocol\": \"amqp091\", \"dest-uri\": \"amqp:///regards.multitenant.manager\", \"dest-queue\": \"${destQueue}\"}"
}

# Param 1 : Shovel ID
function delete_shovel
{
  local shovelId=${1}

  printf >&2 "[\033[32mSHOVEL\033[m]\tRemove shovel ${shovelId}\n"
  rabbitmqctl --quiet clear_parameter shovel migrate-${shovelId}
}


# Param 1 : Queue Name to delete
# Ensure queue is empty
function delete_empty_queue
{
  local queueName=${1}

  printf >&2 "[\033[32mQUEUES\033[m]\tRemove queue ${queueName}\n"
  rabbitmqctl --quiet delete_queue --vhost=regards.multitenant.manager ${queueName} --if-unused --if-empty || printf >&2 "[\033[34mQUEUES\033[m]\tQueue ${queueName} not existing, ignoring error...\n"
}


add_shovel 01 \
   tmp.WorkerManagerResponse \
   regards.worker.manager.response

add_shovel 02 \
  tmp.NotificationRequestEventHandler \
  regards.broadcast.rs-notifier.fr.cnes.regards.modules.notifier.service.flow.NotificationRequestEventHandler

add_shovel 03 \
  tmp.SpecificRecipientNotificationRequestEventHandler \
  regards.broadcast.rs-notifier.fr.cnes.regards.modules.notifier.service.flow.SpecificRecipientNotificationRequestEventHandler

printf >&2 "[\033[32mSHOVEL\033[m]\tWait 30s to ensure messages have been moved from tmp queues to real queues\n"
sleep 30


delete_shovel 01
delete_shovel 02
delete_shovel 03

delete_empty_queue tmp.WorkerManagerResponse
delete_empty_queue tmp.NotificationRequestEventHandler
delete_empty_queue tmp.SpecificRecipientNotificationRequestEventHandler

• Execute the script:

chmod +x /var/lib/rabbitmq/mnesia/migration-to-2.0-restore-messages.sh
/var/lib/rabbitmq/mnesia/migration-to-2.0-restore-messages.sh
rm -f /var/lib/rabbitmq/mnesia/migration-to-2.0-restore-messages.sh

Notifier plugin configuration

info

This migration is only mandatory for REGARDS project using dissemination-ack-sender-plugin.

Notifier dissemination-ack-sender-plugin configuration has changed with the configuration version 2.0 of the plugin. To update the configuration of you plugin, you have to download the plugin configuration from REGARDS HMI update with here under information and export it again.

New recipient label parameter : Add the new recipientLabel parameter in the plugin configuration section. The value of the parameter should be set to the name of the dissemination recipient configured in the notifier sender plugin from the source REGARDS.

{
  "configuration": [
    {
      "key": "fr.cnes.regards.framework.modules.plugins.domain.PluginConfiguration",
      "value": {
        "pluginId": "DisseminationAckSender",
        "parameters": [
          {
            "name": "recipientLabel",
            "type": "STRING",
            "value": "<Ack catalog owner name>",
            "dynamic": false
          },
          ...
        ]
      }
    }
  ]
}

New AIP exchange parameter : If the product to ack in the source REGARDS catalog is an OAIS product, add the RabbitMQ exchange name parameter.

{
  "configuration": [
    {
      "key": "fr.cnes.regards.framework.modules.plugins.domain.PluginConfiguration",
      "value": {
        "pluginId": "DisseminationAckSender",
        "parameters": [
          {
            "name": "aipDisseminationExchange",
            "type": "STRING",
            "value": "<Exchange name to send AIP ack dissemination notification>",
            "dynamic": false
          },
          ...
        ]
      }
    }
  ]
}

New GeoJson exchange parameter : If the product to ack in the source REGARDS catalog is an OAIS product, add the RabbitMQ exchange name parameter.

{
  "configuration": [
    {
      "key": "fr.cnes.regards.framework.modules.plugins.domain.PluginConfiguration",
      "value": {
        "pluginId": "DisseminationAckSender",
        "parameters": [
          {
            "name": "featureDisseminationExchange",
            "type": "STRING",
            "value": "<Exchange name to send AIP ack dissemination notification>",
            "dynamic": false
          },
          ...
        ]
      }
    }
  ]
}

Problems you may encounter

Plugin issue

In the event of a microservice startup error with a Not a managed type: class fr.cnes.regards.modules.xxxx error, make sure there are no plugins from the old version left in the associated NFS directory. If so, delete the old versions manually.

RabbitMQ migration issue

In the event of a microservice startup error with a Going to exit <#20f786a8> c.r.c.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'x-dead-letter-exchange' for queue 'xxx.xxx.xxxx' in vhost 'regards.multitenant.manager': received the value 'regards.DLX' of type 'longstr' but current is none) error, it is necessary to replay the RabbitMQ migration scripts described above.