Full Text Searches

Full text searches may be performed on the data stored within the attached Tenable Log Correlation Engine servers. When viewing the events page the Search field will accept text strings as valid search criteria. Search terms are not case sensitive and a Boolean search may be utilized to further enhance search results. This enables searching the raw logs for details contained in the events.

Tenable Log Correlation Engine can search for compound groups of full text tokens.

Tokens

A token in a full text search is a full word (three or more characters) separated by punctuation or whitespace. For example, if you want to search for logs containing "Microsoft," then Microsoft would be the example of the token.

Operators

Operators are case sensitive, and must be capitalized. For example, a search for mike or miked will actually yield mike AND or AND miked. Multiple operators can be used in a single query.

Operator Description

AND

Finds logs both directly preceding token and the directly following token.

OR

Finds logs containing the directly preceding token, the directly following token, or both.

NOT

Finds logs that do not include the subsequent token.

Wildcards

The * wildcard metacharacter can be used to search for log entries that begin with the token preceding the wildcard in your query. For example, if a wildcard immediately follows a token T, then Tenable Log Correlation Engine will match logs containing a token that starts with T. So, the query text='atten*' will match logs containing "attention," "attenuate," or "Attenborough."

Grouping

Parentheses may be used to group conditionals together to show evaluation precedence just as in mathematics. This is useful in compound conditionals. Without grouping, the query text='blocked AND denied AND dropped OR firewall' would return any log with just “firewall” in it because it satisfies the entire query.

The following query would provide a more accurate result: text='blocked AND denied AND (dropped OR firewall)'

This requires that the log contains blocked, denied, and either dropped or firewall, because it has additional constraints.

Token Adjacency

The relative position of tokens in a log does not normally impact the query results. For example, the query text='video upload' will match both "video staging upload success" and "failed to upload video." If you wish to only match immediately adjacent tokens, surround them with quotation characters ". The query text='"video upload"' will not match "video staging upload success" or "failed to upload video," but it will match "video upload complete."

If only some of the tokens in your query need to be adjacent, you can surround those tokens with parentheses. For example, if you want to search for logs where "upload" immediately follows "video," while "studio" can be anywhere, you can query text='studio ("video upload")' or text='("video upload") studio'.

Token-adjacent search is disabled by default. Enabling token-adjacent search results in a 10% to 15% increase in disk space needed for the database indexes on event log text.

To enable token-adjacent search, run the following command:

/opt/lce/tools/cfg-utils --set-sv position_sensitive_text_search true

Punctuation

Punctuation characters are normally treated as if they were spaces, separating tokens. The ts-test utility, when invoked as ts-test 'bunnies?possibly!', tells us that two asciiword tokens are extracted: bunnies and possibly.

However, if a string looks like a protocol prefix, email address, network name, URL fragment, or file system path, it will be parsed specially.

For more information about the ts-test utility, see ts-test.

Input Output You May Have Expected Actual Output
bunnies:// 1 token: bunnies, of type asciiword 1 token:  bunnies://, of type protocol
mystery.localhost 2 tokens: mystery and localhost, both of type asciiword 1 token:  mystery.localhost, of type host 
[email protected] 3 tokens: bunnies and mystery and localhost, all of type asciiword 1 token:  [email protected], of type email 
I forget which chapter/page.Hmm! 4 tokens: forget and chapter and page and hmm, all of type asciiword 2 tokens: forget, of type asciiword; and chapter/page.hmm, of type file 

Search Query Examples:

Query String What It Means Example Result Example Non-Result Why It Didn't Match

text='Heartbeat'

Show me logs with the term "Heartbeat"

LCE Client Heartbeat| 07/23/2014 00:25:00 AM Hostname: lce_demo IP: 192.0.2.106 Revision: LCE Client 4.2.0 build 20131004

Heart

does not contain the full term "Heartbeat" by itself, only as a substring

text='linux process'

Show me logs with the term "linux" and the term "process"

This linux host executed process "ls". 

This linux host executed nothing.

missing "process"

text='linux NOT process'

Show me logs with the term "linux" but NOT the term "process"

This linux host executed nothing.

This linux host executed process "ls". 

contains "process"

text='linux OR nothing'

Show me logs with either term "linux" or term "nothing"

This linux host executed process "ls".

 

This linux host executed nothing.

This nix host did everything.

does not contain "linux" and does not contain "nothing"

text='(linux OR nothing) AND process'

Show me logs that have terms "linux" and "process" or "nothing" and "process"

This linux host executed process "ls".

 

The process did nothing.

This process did everything.

 

This linux host did nothing.

contains "process" but not "linux" and not "nothing"

 

contains "linux" and "nothing" but not "process"