Extract hostnames and fully qualified domain names from log entries.

Simple hostname:

regex(regex="host[=:]?\s*(?<hostname>[\w-]+)")

Fully qualified domain name (FQDN):

regex(regex="host[=:]?\s*(?<fqdn>[\w-]+\.[\w.-]+)")

Extract IPv4 and IPv6 addresses from log files.

IPv4 address:

regex(regex="(?<ipv4>\b(?:\d{1,3}\.){3}\d{1,3}\b)")

IPv6 address:

regex(regex="(?<ipv6>(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4})")

Extract file paths and file names from logs.

Linux/Unix path:

regex(regex="(?<filepath>/[\w/.-]+)")

Windows path:

regex(regex="(?<filepath>[A-Za-z]:\\\\[\w\\\\.-]+)")

Windows path (excluding spaces and quotes):

regex(regex="(?<proc_path>[A-Za-z]:\\\\[^\\s\"]+)")

This enhanced Windows path pattern uses [^\\s\"]+ to match any character except spaces and quotes. This prevents over-matching in logs where paths might be followed by arguments or other quoted strings. More robust for production environments where paths appear in command lines or structured text.

File extension:

regex(regex="\.(?<extension>log|txt|exe|dll|conf|xml|json)\b")

File name from path:

regex(regex="[\\/](?<filename>[\w.-]+\.\w+)$")

Extract process names and executable names from logs.

Windows executable:

regex(regex="(?<process>\w+\.exe)\b")

Unix process name:

regex(regex="process[=:]?\s*(?<process>[\w.-]+)")

Match reconnaissance tools (using alternation and anchors):

FileName = /^(net|ipconfig|whoami|quser|ping|netstat|tasklist|hostname)\.exe$/i

This pattern matches common reconnaissance tool executables used in security investigations. The components:

The anchors prevent partial matches. For example, the pattern matches net.exe but does not match dotnet.exe because ^ requires the match to start at the beginning of the field.

Windows Event ID:

regex(regex="EventID=(?<event_id>\d+)")

Extracts Windows Event IDs from Windows Event Logs. Common in security monitoring and system administration for tracking specific event types like logon events (4624), account creation (4720), or security audit failures (4625).

Extract usernames in different formats from authentication logs.

Email format (user@domain):

regex(regex="(?<username>[\w.-]+@[\w.-]+)")

Domain format (DOMAIN\user):

regex(regex="(?<domain>[\w-]+)\\\\(?<username>[\w.-]+)")

Simple username:

regex(regex="user[=:]?\s*(?<username>[\w.-]+)")

Email with separate username and domain capture:

regex(field=email, regex="(?<username>[^@]+)@(?<domain>.+)")

This pattern splits an email address into two separate fields: username (everything before the @) and domain (everything after). The [^@]+ matches any character except @, ensuring it stops at the @ symbol.

Extract log severity levels from application logs.

regex(regex="(?<log_level>ERROR|WARN|WARNING|INFO|DEBUG|TRACE|FATAL)\b")

Extract timestamps in common log formats.

ISO8601 format (2024-01-15T10:30:45):

regex(regex="(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})")

Syslog format (Jan 15 10:30:45):

regex(regex="(?<timestamp>(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})")

Apache/NCSA format (15/Jan/2024:10:30:45):

regex(regex="(?<timestamp>\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2})")

Date only - YYYY-MM-DD format (ISO date):

regex(regex="(?<date>\d{4}-\d{2}-\d{2})")

Date only - MM/DD/YYYY format (US date):

regex(regex="(?<date>\d{2}/\d{2}/\d{4})")

Date only - DD-MM-YYYY format (European date):

regex(regex="(?<date>\d{2}-\d{2}-\d{4})")

Hour (24-hour format, 00-23):

regex(regex="(?<hour>[01]\d|2[0-3]):")

This pattern matches hours from 00 to 23. [01]\d matches 00-19, and 2[0-3] matches 20-23.

Minutes (00-59):

regex(regex=":(?<minutes>[0-5]\d):")

Seconds (00-59):

regex(regex=":(?<seconds>[0-5]\d)\b")

Milliseconds (3 digits):

regex(regex="\.(?<milliseconds>\d{3})")

Complete time (HH:MM:SS.mmm):

regex(regex="(?<time>(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d\.\d{3})")

Extract URLs, domains, and API endpoints from web logs.

Full URL:

regex(regex="(?<url>https?://[\w.-]+(?:/[\w/.-]*)?)")

Domain name:

regex(regex="(?<domain>(?:[\w-]+\.)+[a-zA-Z]{2,})")

URI path:

regex(regex="(?:GET|POST|PUT|DELETE|PATCH)\s+(?<uri>/[\w/.-]*)")

Domain from URL with protocol (from referer logs):

regex(regex="https?:\\/\\/(?<domain>.+?)(\\/|\\?|$)", field=referer)

This pattern extracts the domain from a full URL by matching everything after http:// or https:// until the first slash, question mark, or end of string. The .+? uses non-greedy matching to stop at the first delimiter. This pattern is used in the Apache HTTP Server package for referer analysis.

Domain from URL field (without protocol):

regex(regex="^(?<domain>.*?)(\\/|$)", field=url.original, strict=false)

Extracts the domain portion when the URL field does not include the protocol. Matches from the start (^) until the first slash or end of string. This pattern is used in the Zscaler package for web activity analysis.

File extension from URL path:

regex(regex="\\.(?<media_type>[A-Za-z]+)", field=url)

Extracts the file extension after the last dot in a URL path. Useful for categorizing requests by media type (html, json, pdf, and so on). This pattern is used in the Apache HTTP Server package for media type analysis.

URL path components (category and ID):

regex(regex="/products/(?<category>.+)/(?<productId>.+) HTTP/")

Extracts multiple components from a URL path in one pattern. This example captures both the category and product ID from paths like /products/electronics/12345 HTTP/1.1. Useful for analyzing API endpoint usage.

Extract HTTP-related information from web server logs.

HTTP status code:

regex(regex="status[=:]?\s*(?<status_code>[1-5]\d{2})")

HTTP method:

regex(regex="(?<http_method>GET|POST|PUT|DELETE|PATCH|HEAD|OPTIONS)\b")

Extract port numbers from network logs.

regex(regex="port[=:]?\s*(?<port>\d{1,5})")

Port from IP:port format:

regex(regex="(?:\d{1,3}\.){3}\d{1,3}:(?<port>\d{1,5})")

Extract universally unique identifiers and cryptographic hashes from logs.

UUID/GUID:

regex(regex="(?<uuid>[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})")

MAC address (colon separator):

regex(regex="(?<mac>(?:[0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2})")

MAC address (hyphen separator):

regex(regex="(?<mac>(?:[0-9a-fA-F]{2}-){5}[0-9a-fA-F]{2})")

MAC address (either format):

regex(regex="(?<mac>(?:[0-9a-fA-F]{2}[:-]){5}[0-9a-fA-F]{2})")

Email address:

regex(regex="(?<email>[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})")

MD5 hash (32 hexadecimal characters):

regex(regex="(?<md5>\b[a-fA-F0-9]{32}\b)")

SHA1 hash (40 hexadecimal characters):

regex(regex="(?<sha1>\b[a-fA-F0-9]{40}\b)")

SHA256 hash (64 hexadecimal characters):

regex(regex="(?<sha256>\b[a-fA-F0-9]{64}\b)")

Generic hash with field name:

regex(regex="(?:md5|sha1|sha256|hash)[=:]?\s*(?<hash>[a-fA-F0-9]{32,64})")

This pattern matches hashes preceded by common field names like md5, sha1, sha256, or hash. The {32,64} quantifier matches hashes between 32 and 64 characters, covering MD5, SHA1, and SHA256.

Extract version numbers from application logs.

Semantic version (1.2.3):

regex(regex="(?<version>\d+\.\d+\.\d+)")

Version with prefix (v2.4.1):

regex(regex="v(?<version>\d+\.\d+\.\d+)")

Integer part of decimal number:

regex(regex="(?<integer>\d+)\\..*", field=value)

Extracts only the digits before the decimal point from a decimal number. For example, from 3.14159, this pattern captures 3. The \\. matches the literal decimal point, and .* matches any remaining digits after it.

Extract error codes, identifiers, and structured codes from log messages.

Error code with prefix:

regex(regex="(?<error_code>ERR-\d+)", field=message)

Extracts error codes with a specific prefix pattern. This example matches codes like ERR-123, ERR-4567. Combine with a status code filter for targeted error analysis:

status = /^5\d{2}/
| regex(regex="(?<error_code>ERR-\d+)", field=message)

This filters for 5xx HTTP status codes, then extracts the error code from the message field.

Extract syslog-specific fields and encoded data.

Syslog priority (facility and severity):

regex(regex="<(?<priority>\d{1,3})>")

The priority value encodes both facility and severity. To extract them separately, use facility = priority / 8 and severity = priority % 8 after extraction.

Base64 encoded string:

regex(regex="(?<base64>[A-Za-z0-9+/]{20,}={0,2})")

This pattern matches Base64 strings of at least 20 characters (adjust {20,} as needed). The ={0,2} matches the optional padding at the end.

Extract key-value pairs from structured and semi-structured logs.

Key-value pairs (quoted or unquoted values):

regex(regex="(?<key>\w+)=(?<value>\"[^\"]*\"|\S+)")

This pattern extracts key-value pairs in formats like user="john doe" or status=active. The pattern components:

Use this pattern repeatedly to extract multiple key-value pairs from logs formatted as key1=value1 key2="value 2" key3=value3.

Extract content between common delimiters.

Between square brackets:

regex(regex="\[(?<content>[^\]]+)\]")

Between double quotes:

regex(regex="\"(?<content>[^\"]+)\"")

Between parentheses:

regex(regex="\((?<content>[^\)]+)\)")

Between slash and at-sign (command line parsing):

regex(regex="/(?<command>[^@]+)@", field=CommandLine)

Extracts text between / and @ characters. Useful for parsing command line arguments or structured text with specific delimiters. The [^@]+ matches any character except @, capturing everything until the closing delimiter.

Between custom characters:

regex(regex="/\\\/(?<content>.*)@", field=CommandLine)

Extracts text between \/ and @. The pattern uses \\\/ to match the literal characters \/ (backslash-escaped forward slash), then captures everything with .* until reaching @. Adapt this pattern by replacing the start and end delimiters with your specific characters.

These examples show how to parse Apache access logs using regex patterns. Apache supports two common log formats: Common Log Format (CLF) and Combined Log Format.

Apache Common Log Format (simpler):

regex(regex="(?<client_ip>[\d.]+) \S+ \S+ \[(?<req_time>[^\]]+)\] \"(?<method>\w+) (?<path>\S+) \S+\" (?<status>\d{3}) (?<bytes>\d+)")

This pattern extracts core fields from Apache Common Log Format:

192.168.1.100 - - [10/Oct/2023:13:55:36 -0700] "GET /api/users HTTP/1.1" 200 1234

Use this lighter pattern when you do not need referer or user agent information.

Apache Combined Log Format (comprehensive):

regex(regex="^(?<server_name>\S+) (?<client>\S+) \S+ (?<user_name>\S+) \[(?<timestamp>[^\]]+)\] \"(?<method>\S+) (?<url>\S+) HTTP/(?<http_version>[\d.]+)\" (?<status_code>\d{3}) (?<response_size>\d+) \"(?<referer>[^\"]*)\" \"(?<user_agent>[^\"]*)\"")

This pattern extracts all fields from Apache Combined Log Format, including referer and user agent:

example.com 192.168.1.100 - john [10/Oct/2023:13:55:36 -0700] "GET /api/users HTTP/1.1" 200 1234 "https://example.com/home" "Mozilla/5.0"

Pattern breakdown:

Key regex techniques demonstrated:

This example shows how to parse structured application logs that follow a consistent format with timestamp, log level, and message components.

Structured log regex:

regex(regex="(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (?<level>\w+) (?<message>.+)")

This pattern extracts fields from structured application logs in the format:

2024-03-15 14:32:01 ERROR Database connection failed: timeout after 30s
2024-03-15 14:32:05 INFO Retrying connection attempt 2 of 3
2024-03-15 14:32:10 WARN High memory usage detected: 87%

Pattern breakdown:

This pattern works well for application logs that follow a consistent structure. After extraction, you can filter by log level, search within messages, or analyze patterns over time.

Common regex patterns that can cause performance issues or incorrect matches in LogScale.

Avoid: Greedy quantifiers with overlap

// BAD - causes excessive backtracking
regex(regex=".*error.*")

Instead, use non-greedy quantifiers or be more specific:

// BETTER - non-greedy
regex(regex=".*?error")

// BEST - specific pattern
regex(regex="\[(?<level>ERROR)\]")

Avoid: Back references

// BAD - back references are not supported efficiently
regex(regex="(\w+).*\1")

Back references like \1 cause performance issues. Find alternative approaches using field comparisons after extraction.

Avoid: Unnecessary wildcards at start and end

// BAD - implicit wildcard already exists
regex(regex="^.*error.*$")

LogScale implicitly adds .*? to patterns, so adding your own is redundant:

// BETTER - let LogScale handle the implicit wildcard
regex(regex="error")

Avoid: Overlapping character classes

// BAD - \w already includes digits
regex(regex="[\w\d]+")

// BETTER - use only what you need
regex(regex="\w+")

Avoid: Catastrophic backtracking patterns

// BAD - nested quantifiers cause exponential backtracking
regex(regex="(a+)+b")

// BETTER - simplify to single quantifier
regex(regex="a+b")

For more detailed guidance on avoiding regex pitfalls, see Best Practice: Regular Expressions and their Pitfalls.

For more information about using regular expressions in LogScale, see: