Backup of indices

If OpenSearch indices are created in your own bundles, you may also want to create backup copies of these. This can be realized by implementing the interface de.virtimo.bpc.api.BackupSupport.

This offers the following advantages

some boilerplate code is avoided
can be configured so that these backups are created periodically
configuration is carried out via the BPC interface (backup jobs)
the indices can be restored or deleted via the BPC interface

Module-specific backup settings

The backup functionality requires some information for the execution of the backups. This is provided via de.virtimo.bpc.api.BackupSetting.

The following is an explanation of the Parameters:

enabled - true|false depending on whether the backups are to be executed or not
intervalInSeconds - the interval in seconds at which the backups are to be executed. With an interval of 86400 seconds, a backup of the indices is created every 24 hours.
keepBackupsDurationInSeconds - specifies how long the backups are to be kept. Backups older than 90 days are deleted at a specified interval of 7776000 seconds.
indicesToBackup - the names (aliases) of the OpenSearch indices to be backed up. These are backed up together in a so-called snapshot.

The values enabled, intervalInSeconds and keepBackupsDurationInSeconds serve as default values for the backup job that is created from them and can then be adjusted via the BPC interface.

Implementation

In the following ExampleBaseModule.java example, the three indices dein-index-name-1, dein-index-name-2 and dein-index-name-3 are communicated to the BPC Core as indices to be backed up. By default, these should be backed up every 24 hours (= 86400 seconds) and stored for a maximum of 90 days (= 7776000 seconds).

ExampleBaseModule.java

package com.company.example;

import de.virtimo.bpc.api.BackupSetting;
import de.virtimo.bpc.api.BackupSupport;
import de.virtimo.bpc.api.ModuleManager;
import de.virtimo.bpc.api.ModuleConfiguration;
import de.virtimo.bpc.api.ModuleInstance;
import de.virtimo.bpc.module.AbstractInstantiableModule;
import de.virtimo.bpc.module.ModuleConfigurationBuilder;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;

public abstract class ExampleBaseModule extends AbstractInstantiableModule implements BackupSupport {
    private static final Logger LOG = Logger.getLogger(ExampleBaseModule.class.getName());

    public ExampleBaseModule(ModuleManager moduleManager) {
        super(moduleManager);
    }

    // ................

    @Override
    public BackupSetting getBackupSetting() {
        return BackupSetting
                .withEnabled(true)
                .withIntervalInSeconds(86400)
                .withKeepBackupsDurationInSeconds(7776000)
                .withIndicesToBackup("your-index-name-1", "your-index-name-2", "your-index-name-3")
                .build();
    }
}

Was muss bei Wiederherstellung beachtet werden?

Wird ein Backup wiederhergestellt, dann sollten in dieser Zeit keine Daten in die Indices geschrieben werden. Das dient dazu um Datenverlust und evtl. falsch angelegte OpenSearch Indices (kein Alias, falsche Index Einstellungen, falsches Index Mapping) zu vermeiden.

Wird das Schreiben durch REST-Endpunkte initiiert, dann kann man gleich dort einen Fehler zurückliefern oder man erstellt sich eine Queue und arbeitet diese ab, nachdem die Indices vom Backup wiederhergestellt wurden.

Wird vom BPC ein Backup wieder eingespielt, dann werden zuvor und danach Events gesendet auf die man im eigenen Modul reagieren kann. Folgend wird die Registrierung an einem Beispiel gezeigt.

package com.company.example;

import de.virtimo.bpc.api.BackupSetting;
import de.virtimo.bpc.api.BackupSupport;
import de.virtimo.bpc.api.EventRegistration;
import de.virtimo.bpc.api.ModuleManager;
import de.virtimo.bpc.api.ModuleConfiguration;
import de.virtimo.bpc.api.ModuleInstance;
import de.virtimo.bpc.module.AbstractInstantiableModule;
import de.virtimo.bpc.module.ModuleConfigurationBuilder;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;

public abstract class ExampleBaseModule extends AbstractInstantiableModule implements BackupSupport {
    private static final Logger LOG = Logger.getLogger(ExampleBaseModule.class.getName());

    private static final String INDEX_NAME = "dein-index-name";

    private final EventRegistration eventRegistration;

    public ExampleBaseModule(ModuleManager moduleManager) {
        super(moduleManager);

        this.eventRegistration = new EventRegistration(null);
    }

    @Override
    public void setModuleBundle(Bundle moduleBundle) {
        super.setModuleBundle(moduleBundle);

        BundleContext bundleContext = moduleBundle.getBundleContext();

        // register the event handlers
        eventRegistration.setBundleContext(bundleContext);
        eventRegistration.forRestoreBackupStartEvents(new RestoreBackupStartEventHandler());
        eventRegistration.forRestoreBackupDoneEvents(new RestoreBackupDoneEventHandler());
    }

    @Override
    public void destroy() {
        super.destroy();

        // unregister the event handlers
        eventRegistration.unregisterAllEventHandler();
    }

    private class RestoreBackupStartEventHandler extends AbstractRestoreBackupStartEventHandler {
        @Override
        public void processRestoreBackupStartEvent(RestoreBackupStartEvent restoreBackupStartEvent) {
            if (restoreBackupStartEvent.containsAlias(INDEX_NAME)) {
                // e.g. set a flag that a restore of your indices started
            }
        }
    }

    private class RestoreBackupDoneEventHandler extends AbstractRestoreBackupDoneEventHandler {
        @Override
        public void processRestoreBackupDoneEvent(RestoreBackupDoneEvent restoreBackupDoneEvent) {
            if (restoreBackupDoneEvent.containsAlias(INDEX_NAME)) {
                // e.g. set a flag that the restore of your indices is no longer processing
            }
        }
    }
}

Keywords: