Log Correlation Engine 6.0.5 Release Notes (06-30-2020)

New Features

Offline plugins updates can now be accomplished from command line: download and copy lce-combined.tar.gz to path you have designated with the new configuration attribute offline_plugin_update__tarball_path, then restart the lce_www service.
To aid troubleshooting at HA (high availability) sites, --track-packet-counts option has been added to the ha-manager utility.
Full list of all user accounts (both Web UI and nologin), helpful in some troubleshooting scenarios, is now available with user-utils --list-all.
The output of cfg-utils --describe <attributeName> now includes the given attribute's install-time default value.
New show-config--mv--event_rules.sql script, for much improved display of event_rules attribute from command line and in diag reports.
New useful-and-idle--plugins.sql script, to report which PRM and TASL plugins are actively contributing, and how many events have been affected by each. Should you wish to disable in configuration the non-contributing ("idle") plugins, useful-and-idle--plugins.sql makes it easy: it prints an appropriate shell command, which you need only to copy-paste to console.
New scripts have been added to help you need to pinpoint the day and hour when a particular source stopped sending data to LCE Server or compare event traffic volume across LCE Client sources. This suite of SQL scripts is divided into two groups:
- To generate a temporary lookup table: influx-trends--by--src-ip.sql and influx-trends--by--sensor.sql (both parameterized by a date range); and influx-trends--by--compound-profile--clients-only.sql (parameterized by silo#).
- To query a temporary lookup table: _i_t_q__{hourly,daily,weekly,monthly}__{absolute,baseline_start,baseline_finish,coevals_rank,delta_immediately_previous}.sql
The date and time of the last successful login are now displayed in Web UI after login.

Changed Functionality and Performance Enhancements

Maximum number of LCE Clients allowed to connect concurrently is now 20476, up from 8192.
Average time to match a log against event rules has been roughly halved; customers with significant number of configured event rules may note increased throughput.
The reset-account utility has been renamed to user-utils to reflect the broader functionality it now encompasses; it is located in the same /opt/lce/tools directory.
The cfg-utils utility now performs even more input validation: for example, it will check an attempt to assign same port number to 2+ port number configuration attributes.
The optimize-datastore utility now needs 5-8% less time to process a silo, in the default (i.e. neither --also-cluster nor --also-reindex) operating mode.

Bug Fixes

After the lce_client service in Windows LCE Client installs restarted, in certain circumstances LCE Server would mistakenly create duplicate roster entries representing that client.
Deleting a large number (over 200) clients through Web UI could cause the lced daemon to terminate abnormally.
In certain circumstances which can arise particularly at high-volume sites, operation of the optimize-datastore utility could interfere with silo roll process, resulting in oversize silos and/or overall performance drop.
An invalid event rule was accepted partially, instead of being rejected wholly, and such an incomplete rule would interrupt event normalization.
Failure to run the PostgreSQL VACUUM command on pm_clients__ephemera table could impact response Web UI time with some unusual workloads.
Output of cfg-utils --get intrusion_detect_sensors did not include the customer_codes__comma_sep attribute.
Values in several columns of status_client_counts DB table, and Web UI pages populated therefrom, could diverge from expected.
When the lce_server service is started from command line (whether directly or via start-all script or restart-all script), a few trace messages intended for lced tracelog are printed to console; this may give false impression that startup has not completed successfully.
Adding a Web UI user from Web UI would add an account with username 0, regardless of the username specified by operator.
Web UI link for changing one's own password did not work consistently.
Web UI controls for locking / unlocking a Web UI user did not function, for a particular subset of LCE Server installations.
The minimum edit distance constraint, as configured by the web_UI__password__minimum_edit_distance attribute, was not enforced when changing one's own password in Web UI.
The Occasion column in Alerts tab of Web UI was too narrow to accommodate some alert occasion names; in such cases, a single alert would take up extra vertical display space, reducing the overall number of alerts visible without scrolling.

Security Enhancements

Internal Audit Log for Actions Pertaining to Web UI User Accounts

All account-admin actions (add acct, unlock acct, etc.) and session-scope actions (begin session, end session, etc.) with failure outcome are now tracked in this audit log.
If so configured, LCE will also track session-scope actions (begin session, end session, etc.) with success outcome.
If so configured, LCE will copy each audit entry to host syslog in real-time.
If so configured, LCE will, furthermore, dumped this audit log to a backup location of your choice at an interval of your choice.

Enforce Your Site's Password Reuse Policy

The password provided when adding a Web UI user account is now always considered a temporary password and the account owner must change it on first login.
If configured, LCE will now enforce minimum and/or maximum password lifetime.
If configured, LCE will now prevent a Web UI user from changing password to one used within the latest n password changes.
If configured, LCE will now prevent a Web UI user from changing password to one differing by fewer than n characters (per Levenshtein edit distance) from the current password.

Enforce Your Site's Login Session Policy

If configured, LCE will now automatically lock the account of a Web UI user who has attempted but failed to login at least m times within n consecutive minutes.
If configured, LCE will now automatically lock the account of a Web UI user who has not been active within n consecutive hours.

Better Control of LCE Clients

If configured, LCE will now automatically logout and revoke the authorization of a client which has been inactive within n contiguous days.

Upgrade Notes

If you are upgrading from a version earlier than LCE 4.8.4, upgrade to LCE 4.8.4 before upgrading to LCE 6.0.5.
If you are upgrading from LCE 5.0.x, upgrade to LCE 5.1.1 before upgrading to LCE 6.0.5.
If you are upgrading from LCE 4.8.4 with high availability configured, use the method described in Migrate Your High Availability Configuration to LCE 6.0.4 or Later in the LCE User Guide.
If you are upgrading from 6.0.4 with high availability configured, use the method described in Migrate Your High Availability Configuration to LCE 6.0.5 or Later in the LCE User Guide.
If you are upgrading from LCE 5.x.y or 6.0.0, run:

rpm --nopreun -Uvh lce-6.0.5-el6.x86_64.rpm
If you are upgrading from LCE 6.0.x to LCE 6.0.5, after the rpm command completes, run:

nohup /opt/lce/tmp/upgr603-rebuild-silos &

This command rebuilds your pre-existing event silos in the new format (which takes up less disk space and improves query performance). As each silo is rebuilt, it will automatically become available for querying again. The upgr603-rebuild-silos script will take 25-30 minutes to rebuild each pre-existing silo; it prioritizes silos with the most recent events.
If you are prompted to run /opt/lce/tmp/restore_per-client_decisions.sql and you performed explicit client authorizations (without the aid of client assignment rules or the auto-authorization setting) or specified custom sensor names for individual LCE clients, run:

/opt/lce/tmp/restore_per-client_decisions.sql

This script applies the changes you made to your upgraded LCE.

Note: If you did not perform explicit client authorizations or specify custom sensor names for individual LCE clients, you do not need to run /opt/lce/tmp/restore_per-client_decisions.sql