Backing up indices

If OpenSearch indices are created in custom bundles, you may want to create backup copies of them as well. This can be achieved by implementing the interface de.virtimo.bpc.api.BackupSupport.

This offers the following advantages

It avoids a significant amount of boilerplate code
It can be configured to create these backups periodically
Configuration is done via the BPC interface (Backup Jobs)
indices can be restored or deleted via the BPC interface

Module-specific backup settings

The backup functionality requires some information to perform the backups. This is provided via a de.virtimo.bpc.api.BackupSetting.

The following explains the parameters:

enabled - true|false depending on whether the backups should be run or not
intervalInSeconds - the interval in seconds at which the backups should be performed. With an interval of 86400 seconds, a backup of the indexes is created every 24 hours.
keepBackupsDurationInSeconds - specifies how long the backups should be retained. With a retention period of 7776000 seconds, backups older than 90 days are deleted.
indicesToBackup - the names (aliases) of the OpenSearch indices to be backed up. These are backed up together in a so-called snapshot.

The values enabled, intervalInSeconds, and keepBackupsDurationInSeconds serve as default values for the backup job that is created from them and can subsequently be adjusted via the BPC interface.

Implementation

In the following ExampleBaseModule.java example, the three indices dein-index-name-1, dein-index-name-2, and dein-index-name-3 are specified to the BPC Core as indices to be backed up. By default, these are to be backed up every 24 hours (= 86,400 seconds) and retained for a maximum of 90 days (= 7,776,000 seconds).

ExampleBaseModule.java

package com.company.example;

import de.virtimo.bpc.api.BackupSetting;
import de.virtimo.bpc.api.BackupSupport;
import de.virtimo.bpc.api.ModuleManager;
import de.virtimo.bpc.api.ModuleConfiguration;
import de.virtimo.bpc.api.ModuleInstance;
import de.virtimo.bpc.module.AbstractInstantiableModule;
import de.virtimo.bpc.module.ModuleConfigurationBuilder;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;

public abstract class ExampleBaseModule extends AbstractInstantiableModule implements BackupSupport {
    private static final Logger LOG = Logger.getLogger(ExampleBaseModule.class.getName());

    public ExampleBaseModule(ModuleManager moduleManager) {
        super(moduleManager);
    }

    // ................

    @Override
    public BackupSetting getBackupSetting() {
        return BackupSetting
                .withEnabled(true)
                .withIntervalInSeconds(86400)
                .withKeepBackupsDurationInSeconds(7776000)
                .withIndicesToBackup("dein-index-name-1", "dein-index-name-2", "dein-index-name-3")
                .build();
    }
}

What must be considered during restoration?

When restoring a backup, no data should be written to the indices during this time. This is to prevent data loss and potentially incorrectly created OpenSearch indices (no alias, incorrect index settings, incorrect index mapping).

If the write operation is initiated via REST endpoints, you can return an error immediately there, or you can create a queue and process it after the indices have been restored from the backup.

When a backup is restored by the BPC, events are sent both before and after the restoration, to which you can respond in your own module. The following example demonstrates the registration process.

package com.company.example;

import de.virtimo.bpc.api.BackupSetting;
import de.virtimo.bpc.api.BackupSupport;
import de.virtimo.bpc.api.EventRegistration;
import de.virtimo.bpc.api.ModuleManager;
import de.virtimo.bpc.api.ModuleConfiguration;
import de.virtimo.bpc.api.ModuleInstance;
import de.virtimo.bpc.module.AbstractInstantiableModule;
import de.virtimo.bpc.module.ModuleConfigurationBuilder;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;

public abstract class ExampleBaseModule extends AbstractInstantiableModule implements BackupSupport {
    private static final Logger LOG = Logger.getLogger(ExampleBaseModule.class.getName());

    private static final String INDEX_NAME = "dein-index-name";

    private final EventRegistration eventRegistration;

    public ExampleBaseModule(ModuleManager moduleManager) {
        super(moduleManager);

        this.eventRegistration = new EventRegistration(null);
    }

    @Override
    public void setModuleBundle(Bundle moduleBundle) {
        super.setModuleBundle(moduleBundle);

        BundleContext bundleContext = moduleBundle.getBundleContext();

        // register the event handlers
        eventRegistration.setBundleContext(bundleContext);
        eventRegistration.forRestoreBackupStartEvents(new RestoreBackupStartEventHandler());
        eventRegistration.forRestoreBackupDoneEvents(new RestoreBackupDoneEventHandler());
    }

    @Override
    public void destroy() {
        super.destroy();

        // unregister the event handlers
        eventRegistration.unregisterAllEventHandler();
    }

    private class RestoreBackupStartEventHandler extends AbstractRestoreBackupStartEventHandler {
        @Override
        public void processRestoreBackupStartEvent(RestoreBackupStartEvent restoreBackupStartEvent) {
            if (restoreBackupStartEvent.containsAlias(INDEX_NAME)) {
                // e.g. set a flag that a restore of your indices started
            }
        }
    }

    private class RestoreBackupDoneEventHandler extends AbstractRestoreBackupDoneEventHandler {
        @Override
        public void processRestoreBackupDoneEvent(RestoreBackupDoneEvent restoreBackupDoneEvent) {
            if (restoreBackupDoneEvent.containsAlias(INDEX_NAME)) {
                // e.g. set a flag that the restore of your indices is no longer processing
            }
        }
    }
}

Keywords: