V1.14 to V1.15

Steps to migrate REGARDS from version 1.14 to 1.15

Please read until the end

There is some steps remaining at the end of this guide related to RabbitMQ

• Download last playbook version and move your inventory inside the new playbook
• Edit your inventory file group_vars/all/main.yml :

# Replace 
version: 1.14.9
# Into 
version: 1.15.2

info

When this guide has been written, last version was 1.15.2. Check if there is an updated version available here

• Remove deprecated property docker_dns inside your inventory file group_vars/docker_nodes/main.yml:

#docker_dns:
#  - 8.8.8.8

• Inside your inventory file group_vars/regards_nodes/main.yml, remove deprecated properties:

  • all references to enable_resource_limits and enable_resource_limits
  • property group_docker_use_log_concentrator
  • property group_docker_mservices.front.securised

• If you declare a some memory limit override inside your inventory file group_vars/regards_nodes/main.yml, you need to write 2560m instead of 2560mb or 2g instead of 2gb (remove b)

• Inside your inventory file group_vars/regards_nodes/main.yml, add group_docker_static_files_migrated property:

[ ... ]
group_docker_registry: ghcr.io/regardsoss

# Files needs to be moved from ./files/ to ../../static/
# using $ mv ./files ../../static
group_docker_static_files_migrated: False

group_docker_tags:
  [ ... ]

info

Static files will be migrated with REGARDS V1.16 release.

• Shutdown REGARDS using the playbook

ansible-playbook -i [...] regards-shutdown.yml [...]

• Disconnect swarm nodes

ansible-playbook -i [...] delete-swarm.yml [...]

• Reinstall swarm nodes

ansible-playbook -i [...] setup-vm.yml [...]

info

You must do the setup-vm.yml step whether you use the logging stack or not

• Update REGARDS using the playbook

ansible-playbook -i [...] regards.yml [...]

• Wait the stack to be up and running

cd /path/to/regards/cli
./status.sh
# Check every containers is up and running

• Now, connect to your RabbitMQ node or one of its replicate

# Enter inside one of the RabbitMQ container
./exec.sh rs-rabbitmq bash

• Activate two flags required to migrate from RabbitMQ 3.8 to RabbitMQ 3.11 later, with REGARDS V1.16 release:

rabbitmqctl enable_feature_flag maintenance_mode_status
rabbitmqctl enable_feature_flag user_limits

warning

RabbitMQ 3.11 requires these feature flags to be enabled before the upgrade. The upgrade of REGARDS V1.16 will fail if you miss this step.

• Create a migration script using vi /var/lib/rabbitmq/mnesia/migration-to-1.15.sh with following content:

#!/bin/bash -e
ADMIN_USER=guest
ADMIN_PASSWORD=guest

# Param 1 : Shovel ID
# Param 2 : Source queue
# Param 2 : Dest queue
function add_shovel
{
  local shovelId=${1}
  local srcQueue=${2}
  local destQueue=${3}

  printf >&2 "[\033[32mSHOVEL\033[m]\tAdd shovel to move ${srcQueue} to ${destQueue}\n"
  rabbitmqctl --quiet set_parameter shovel migrate-${shovelId} \
  "{\"src-protocol\": \"amqp091\", \"src-uri\": \"amqp:///regards.multitenant.manager\", \"src-queue\": \"${srcQueue}\", \"dest-protocol\": \"amqp091\", \"dest-uri\": \"amqp:///regards.multitenant.manager\", \"dest-queue\": \"${destQueue}\"}"
}

# Param 1 : Shovel ID
function delete_shovel
{
  local shovelId=${1}

  printf >&2 "[\033[32mSHOVEL\033[m]\tRemove shovel ${shovelId}\n"
  rabbitmqctl --quiet clear_parameter shovel migrate-${shovelId}
}


# Param 1 : Queue Name to delete
# Ensure queue is empty
function delete_empty_queue
{
  local queueName=${1}

  printf >&2 "[\033[32mQUEUES\033[m]\tRemove queue ${queueName}\n"
  rabbitmqctl --quiet delete_queue --vhost=regards.multitenant.manager ${queueName} --if-unused --if-empty || printf >&2 "[\033[34mQUEUES\033[m]\tQueue ${queueName} not existing, ignoring error...\n"
}


# Param 1 : Queue Name to delete
# Do not check if queue is empty
function delete_queue
{
  local queueName=${1}

  printf >&2 "[\033[32mQUEUES\033[m]\tRemove queue ${queueName}\n"
  rabbitmqctl --quiet delete_queue --vhost=regards.multitenant.manager ${queueName} --if-unused || printf >&2 "[\033[34mQUEUES\033[m]\tQueue ${queueName} not existing, ignoring error...\n"
}

# Param 1 : Exchange Name to delete
function delete_exchange
{
  local exchangeName=${1}

  printf >&2 "[\033[32mEXCHANGES\033[m]\tRemove exchange ${exchangeName}\n"
  rabbitmqadmin --username=$ADMIN_USER --password=$ADMIN_PASSWORD -V regards.multitenant.manager delete exchange name="${exchangeName}" || printf >&2 "[\033[34mEXCHANGES\033[m]\tExchange ${exchangeName} not existing, ignoring error...\n"
}



add_shovel 01 \
  regards.broadcast.rs-dam.fr.cnes.regards.modules.storage.client.FileRequestGroupEventHandler \
  regards.broadcast.rs-dam.fr.cnes.regards.modules.filecatalog.client.handler.FileRequestGroupEventHandler

add_shovel 02 \
  regards.broadcast.rs-ingest.fr.cnes.regards.modules.storage.client.FileRequestGroupEventHandler \
  regards.broadcast.rs-ingest.fr.cnes.regards.modules.filecatalog.client.handler.FileRequestGroupEventHandler

add_shovel 03 \
  regards.broadcast.rs-fem.fr.cnes.regards.modules.storage.client.FileRequestGroupEventHandler \
  regards.broadcast.rs-fem.fr.cnes.regards.modules.filecatalog.client.handler.FileRequestGroupEventHandler

add_shovel 04 \
  regards.broadcast.rs-order.fr.cnes.regards.modules.storage.client.FileReferenceEventHandler \
  regards.broadcast.rs-order.fr.cnes.regards.modules.filecatalog.client.handler.FileReferenceEventHandler

add_shovel 05 \
  regards.broadcast.rs-order.fr.cnes.regards.modules.storage.client.FileReferenceUpdateEventHandler \
  regards.broadcast.rs-order.fr.cnes.regards.modules.filecatalog.client.handler.FileReferenceUpdateEventHandler

add_shovel 06 \
  regards.broadcast.fr.cnes.regards.modules.workermanager.dto.events.in.RequestEvent \
  regards.broadcast.rs-worker-manager.fr.cnes.regards.modules.workermanager.service.flow.RequestHandler

add_shovel 07 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.AvailabilityFlowItemHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.FileRestorationRequestEventHandler

add_shovel 08 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.CancelRequestGroupsEventHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.CancelRequestGroupsEventHandler

add_shovel 09 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.CopyFlowHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.FilesCopyEventHandler

add_shovel 10 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.DeletionFlowHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.FilesDeletionEventHandler

add_shovel 11 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.ReferenceFlowItemHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.FilesReferenceEventHandler

add_shovel 12 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.RetryFlowItemHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.FilesRetryRequestEventHandler

add_shovel 13 \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.StorageFlowItemHandler \
  regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.handler.FilesStorageRequestEventHandler

printf >&2 "[\033[32mSHOVEL\033[m]\tWait 30s to ensure messages have been moved\n"
sleep 30

delete_shovel 01
delete_shovel 02
delete_shovel 03
delete_shovel 04
delete_shovel 05
delete_shovel 06
delete_shovel 07
delete_shovel 08
delete_shovel 09
delete_shovel 10
delete_shovel 11
delete_shovel 12
delete_shovel 13

delete_empty_queue regards.broadcast.rs-dam.fr.cnes.regards.modules.storage.client.FileRequestGroupEventHandler
delete_empty_queue regards.broadcast.rs-ingest.fr.cnes.regards.modules.storage.client.FileRequestGroupEventHandler
delete_empty_queue regards.broadcast.rs-fem.fr.cnes.regards.modules.storage.client.FileRequestGroupEventHandler
delete_empty_queue regards.broadcast.rs-order.fr.cnes.regards.modules.storage.client.FileReferenceEventHandler
delete_empty_queue regards.broadcast.rs-order.fr.cnes.regards.modules.storage.client.FileReferenceUpdateEventHandler
delete_empty_queue regards.broadcast.fr.cnes.regards.modules.workermanager.dto.events.in.RequestEvent
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.AvailabilityFlowItemHandler
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.CancelRequestGroupsEventHandler
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.CopyFlowHandler
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.DeletionFlowHandler
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.ReferenceFlowItemHandler
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.RetryFlowItemHandler
delete_empty_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.StorageFlowItemHandler

delete_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.ReferenceFlowItemHandler.DLQ
delete_queue regards.broadcast.rs-storage.fr.cnes.regards.modules.storage.service.file.flow.StorageFlowItemHandler.DLQ
delete_queue regards.broadcast.rs-catalog.fr.cnes.regards.modules.catalog.stac.service.collection.IdMappingUpdateHandler
delete_queue regards.unicast.swot.rs-dam.fr.cnes.regards.modules.dam.domain.entities.event.DatasetEvent
delete_queue regards.unicast.swot.rs-dam.fr.cnes.regards.modules.dam.domain.entities.event.NotDatasetEntityEvent

delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.RetryFlowItem
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.AvailabilityFlowItem
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.CopyFlowItem
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.DeletionFlowItem
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.ReferenceFlowItem
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.RetryFlowItem
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.flow.StorageFlowItem

delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.event.CancelRequestEvent
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.event.FileReferenceEvent
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.event.FileReferenceUpdateEvent
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.event.FileRequestsGroupEvent
delete_exchange regards.broadcast.fr.cnes.regards.modules.storage.domain.event.QuotaUpdateEvent

• Edit ADMIN_USER and ADMIN_PASSWORD variables if you disabled the guest default admin user
• Execute the script:

chmod +x /var/lib/rabbitmq/mnesia/migration-to-1.15.sh
/var/lib/rabbitmq/mnesia/migration-to-1.15.sh
rm -f /var/lib/rabbitmq/mnesia/migration-to-1.15.sh

Migrating from 1.15.1 to 1.15.2

If you have installed the 1.15.1 version and want to upgrade to 1.15.X, some microservices won't boot with flyway issues.

• You need to add the following property inside your inventory file group_vars/regards_nodes/main.yml:

group_config_mservices:
  flyway:
    out_of_order: true

• Update REGARDS using the playbook

ansible-playbook -i [...] regards.yml [...]

• Ensure REGARDS is running fine (using status.sh)

• Remove the added property flyway.out_of_order and update REGARDS again.