Silo Archiving

Configuration

• Total size of activeDb is limited by config attribute active-size (default: 20 TB).
• Total size of archiveDb is limited by config attribute archive-size (default: 20 TB).

Control Flow

Every 2.5 minutes, LCE will:

1. Read in the results of the last-executed action, from LCE status database.

2. Choose the next action to take based on the last-executed action.

3. Perform the next action and store results in LCE status database.

Storing the state in this manner has the following advantages:

• simplicity (no separate logic to handle reloads/restarts is needed)

• transparency (to see exactly where the archival algorithm is, just query the LCE status database.)

• available emergency override (can alter the control flow by updating the LCE status database.)

  Note: This is not standard operating procedure and should only be performed in very rare cases.

LCE waits a maximum of 60 minutes for an archive job to complete in order to avoid being stuck in the CheckArchiveDone state indefinitely in the rare case that PostgreSQL fails to report an archive job as complete.