Display Allowed URL Categories And Domains

Visualize relationships between URL categories and domains

This is a query example for the Top Allowed Categories to Domains widget in the Web - User Investigation dashboard of the zscaler/internet-access package.

Query

flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result

logscale

#event.dataset = "zia.web"
| user.email=~wildcard(*, ignoreCase=true)
| regex("^(?<Vendor.domain>.*?)(\/|$)", field=url.original, strict=false)
| event.action="allowed"
| top([Vendor.urlcategory, Vendor.domain], limit=20)
| sankey(source="Vendor.urlcategory", target="Vendor.domain")

Introduction

This widget is used to visualize the relationships between allowed URL categories and their associated domains in Zscaler Internet Access web traffic using a Sankey diagram.

In this widget, the regex() function extracts domain information, the top() function ranks the combinations, and the sankey() function creates a flow diagram showing relationships between categories and domains.

Example incoming data might look like this:

@timestamp	#Cps.version	#Vendor	#ecs.version	#error	#event.dataset	#event.kind	#event.module	#event.outcome	#repo	#type	@error	@error_msg	@error_msg[0]	@event_parsed	@id	@ingesttimestamp	@rawstring	@timezone	Parser.version	Vendor.Recordtype	Vendor.action	Vendor.actiontaken	Vendor.adminid	Vendor.algo	Vendor.applicationname	Vendor.auditlogtype	Vendor.authentication	Vendor.authtype	Vendor.category	Vendor.channel	Vendor.clientip	Vendor.company	Vendor.datetime	Vendor.dept	Vendor.destinationip	Vendor.destinationipend	Vendor.destinationipstart	Vendor.destinationport	Vendor.destinationportstart	Vendor.dlpdictcount	Vendor.dlpdictnames	Vendor.dlpenginenames	Vendor.errorcode	Vendor.filedownloadtimems	Vendor.filemd5	Vendor.filename	Vendor.filescantimems	Vendor.filesource	Vendor.filetypename	Vendor.fullurl	Vendor.ikeversion	Vendor.interface	Vendor.itemdstname	Vendor.lastmodtime	Vendor.lifebytes	Vendor.lifetime	Vendor.location	Vendor.login	Vendor.policy	Vendor.policydirection	Vendor.protocol	Vendor.recordid	Vendor.resource	Vendor.result	Vendor.rulename	Vendor.severity	Vendor.sourceip	Vendor.sourceport	Vendor.sourceportstart	Vendor.sourcetype	Vendor.spi	Vendor.spi_in	Vendor.spi_out	Vendor.srcipend	Vendor.srcipstart	Vendor.subcategory	Vendor.tenant	Vendor.threatname	Vendor.time	Vendor.tunnelprotocol	Vendor.tunneltype	Vendor.user	destination.address	destination.ip	destination.port	event.action	event.category[0]	event.category[1]	event.category[2]	event.id	event.severity	event.type[0]	file.directory	file.extension	file.hash.md5	file.name	group.name	network.direction	network.type	rule.name	source.address	source.geo.name	source.ip	source.port	url.full	url.path	user.domain	user.email	user.name
2026-02-10T06:02:22	1.1.0	zscaler	9.2.0	true	zia.casb	alert	zia		auto-dashboard-queries	zscaler-internetaccess	true	Error parsing timestamp. errormsg="Text '2026-02-10T06:02:21.304Z' could not be parsed at index 0" zone=""	Error parsing timestamp. errormsg="Text '2026-02-10T06:02:21.304Z' could not be parsed at index 0" zone=""	false	RG0lMmagN4Hpu0YtU49sDAs0_3_4_1770703342	2026-02-10T06:02:22	{"sourcetype":"zscalernss-casb","event":{"threatname":"Win32.Emotet","fullurl":"/images/products/electronics/phone-2024.jpg","dlpenginenames":"Credit Card","datetime":"2026-02-10T06:02:21.304Z","filename":"svchost.exe","recordid":"f47ac10b-58cc-4372-a567-0e02b2c3d479","policy":"Corporate Data Protection","dept":"IT","filescantimems":"0","dlpdictnames":"Credit Cards,SSN","company":"Acme Corporation","dlpdictcount":"123400","applicationname":"Salesforce","filesource":"OneDrive","login":"phishing@malicious-domain.com","tenant":"Production","filedownloadtimems":"1","filemd5":"a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0","lastmodtime":"2026-02-10T06:02:21.304Z"}}	Z	4.0.0						Salesforce							Acme Corporation	2026-02-10T06:02:21.304Z	IT						123400	Credit Cards,SSN	Credit Card		1	a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0	svchost.exe	0	OneDrive		/images/products/electronics/phone-2024.jpg				2026-02-10T06:02:21.304Z				phishing@malicious-domain.com	Corporate Data Protection			f47ac10b-58cc-4372-a567-0e02b2c3d479								zscalernss-casb							Production	Win32.Emotet									authentication	file	threat	f47ac10b-58cc-4372-a567-0e02b2c3d479		indicator	OneDrive		a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0	svchost.exe	Acme Corporation			Win32.Emotet					/images/products/electronics/phone-2024.jpg	/images/products/electronics/phone-2024.jpg	malicious-domain.com	phishing@malicious-domain.com	phishing
2026-02-10T06:02:22	1.1.0	zscaler	9.2.0		zia.audit	event	zia	success	auto-dashboard-queries	zscaler-internetaccess					CcdZtVsyi1yvhvYT6sRMG6EV_3_3_1770703342	2026-02-10T06:02:22	{"event":{"clientip":"192.168.2.143","resource":"Firewall Rule","recordid":"6ba7b810-9dad-11d1-80b4-00c04fd430c8","result":"SUCCESS","auditlogtype":"Admin Audit","adminid":"admin@evil-site.net","subcategory":"Firewall Policy","interface":"UI","action":"Create","postaction":{},"preaction":{},"category":"Policy","time":"2026-02-10T06:02:22.099Z","errorcode":"ERR_001"},"sourcetype":"zscalernss-audit"}	Z	4.0.0		Create		admin@evil-site.net			Admin Audit			Policy		192.168.2.143												ERR_001									UI										6ba7b810-9dad-11d1-80b4-00c04fd430c8	Firewall Rule	SUCCESS						zscalernss-audit						Firewall Policy			2026-02-10T06:02:22.099Z							Create	configuration			6ba7b810-9dad-11d1-80b4-00c04fd430c8		creation											192.168.2.143				evil-site.net	admin@evil-site.net	admin
2026-02-10T06:02:23	1.1.0	zscaler	9.2.0		zia.edlp	event	zia		auto-dashboard-queries	zscaler-internetaccess					tDcWan7CVbbOjUEvJaqdrD33_2_4_1770703343	2026-02-10T06:02:23	{"sourcetype":"zscalernss-edlp","event":{"severity":"High","itemdstname":"explorer.exe","filemd5":"9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c3b2a1f0e9d8c7b6a5f4e3d2c1b0a9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c3b2a1f0","dlpdictnames":"PII,PHI","dept":"HR","filetypename":"PDF","dlpdictcount":"456700","login":"support@suspicious-portal.org","rulename":"Block Malware","recordid":"3d6f4e2a-8b9c-4f1e-a2d5-7c8e9f0a1b2c","actiontaken":"Allow","datetime":"2026-02-10T06:02:22.873Z","dlpenginenames":"SSN","channel":"Email"}}	Z	4.0.0			Allow								Email			2026-02-10T06:02:22.873Z	HR						456700	PII,PHI	SSN			9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c3b2a1f0e9d8c7b6a5f4e3d2c1b0a9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c3b2a1f0				PDF				explorer.exe					support@suspicious-portal.org				3d6f4e2a-8b9c-4f1e-a2d5-7c8e9f0a1b2c			Block Malware	High				zscalernss-edlp																Allow	file	network		3d6f4e2a-8b9c-4f1e-a2d5-7c8e9f0a1b2c	70	allowed		PDF	9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c3b2a1f0e9d8c7b6a5f4e3d2c1b0a9f8e7d6c5b4a3f2e1d0c9b8a7f6e5d4c3b2a1f0					Block Malware							suspicious-portal.org	support@suspicious-portal.org	support
2026-02-10T06:02:24	1.1.0	zscaler	9.2.0		zia.tunnel	event	zia		auto-dashboard-queries	zscaler-internetaccess					M0pQsDX2VvpH4yfoFvePp1gB_2_16_1770703344	2026-02-10T06:02:24	{"sourcetype":"zscalernss-tunnel","event":{"sourceip":"192.168.0.87","destinationportstart":"567800","lifebytes":"5372846913","protocol":"HTTP","datetime":"2026-02-10T06:02:23.647Z","authtype":"PSK","ikeversion":"2","destinationipstart":"192.168.2.16","sourceportstart":"234500","spi":"3847562891","srcipend":"192.168.4.198","destinationipend":"192.168.0.234","sourceport":"789300","location":"Seattle","Recordtype":"ike_phase2","srcipstart":"192.168.1.54","tunnelprotocol":"ESP","user":"adamsb","policydirection":"Inbound","recordid":"9e8d7c6b-5a4f-4e3d-2c1b-0a9f8e7d6c5b","lifetime":"4","tunneltype":"IPSEC IKEV 1","destinationip":"192.168.3.211","authentication":"SHA256","algo":"AES-256"}}	Z	4.0.0	ike_phase2				AES-256			SHA256	PSK					2026-02-10T06:02:23.647Z		192.168.3.211	192.168.0.234	192.168.2.16		567800												2				5372846913	4	Seattle			Inbound	HTTP	9e8d7c6b-5a4f-4e3d-2c1b-0a9f8e7d6c5b					192.168.0.87	789300	234500	zscalernss-tunnel	3847562891			192.168.4.198	192.168.1.54					ESP	IPSEC IKEV 1	adamsb	192.168.3.211	192.168.3.211			network			9e8d7c6b-5a4f-4e3d-2c1b-0a9f8e7d6c5b								inbound	ipsec ikev 1		192.168.0.87	Seattle	192.168.0.87	789300					adamsb
2026-02-10T06:02:25	1.1.0	zscaler	9.2.0		zia.tunnel	event	zia		auto-dashboard-queries	zscaler-internetaccess					M0pQsDX2VvpH4yfoFvePp1gB_2_17_1770703345	2026-02-10T06:02:25	{"event":{"Recordtype":"ike_phase1","destinationip":"192.168.1.178","algo":"AES-192","location":"Munich","authentication":"SHA1","sourceport":"890100","datetime":"2026-02-10T06:02:24.417Z","lifetime":"13","spi_in":"2947183746","ikeversion":"2","authtype":"Certificate","tunneltype":"IPSEC IKEV 1","user":"andersonk","destinationport":"345600","sourceip":"192.168.3.45","recordid":"1a2b3c4d-5e6f-4a7b-8c9d-0e1f2a3b4c5d","spi_out":"1928374655"},"sourcetype":"zscalernss-tunnel"}	Z	4.0.0	ike_phase1				AES-192			SHA1	Certificate					2026-02-10T06:02:24.417Z		192.168.1.178			345600													2					13	Munich					1a2b3c4d-5e6f-4a7b-8c9d-0e1f2a3b4c5d					192.168.3.45	890100		zscalernss-tunnel		2947183746	1928374655								IPSEC IKEV 1	andersonk	192.168.1.178	192.168.1.178	345600		network			1a2b3c4d-5e6f-4a7b-8c9d-0e1f2a3b4c5d								unknown	ipsec ikev 1		192.168.3.45	Munich	192.168.3.45	890100					andersonk

Step-by-Step

Starting with the source repository events.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result style 1 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
#event.dataset = "zia.web"
```
Filters events where the #event.dataset field equals zia.web.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result style 2 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| user.email=~wildcard(*, ignoreCase=true)
```
Matches any email address in the user.email field using the wildcard() function. The ignoreCase parameter set to true ensures case-insensitive matching.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result style 3 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| regex("^(?<Vendor.domain>.*?)(\/|$)", field=url.original, strict=false)
```
Extracts the domain from the url.original field using a regular expression pattern and stores it in a new field named Vendor.domain. The strict parameter set to false allows the pattern to match partially.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result style 4 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| event.action="allowed"
```
Filters events where the event.action field equals allowed.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result style 5 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| top([Vendor.urlcategory, Vendor.domain], limit=20)
```
Finds the most frequent combinations of URL categories and domains from the Vendor.urlcategory and Vendor.domain fields, and returns the results in a _count field. The limit parameter restricts the output to 20 entries.
flowchart LR; %%{init: {"flowchart": {"defaultRenderer": "elk"}} }%% repo{{Events}} 1[/Filter/] 2[/Filter/] 3>Augment Data] 4[/Filter/] 5{{Aggregate}} 6{{Aggregate}} result{{Result Set}} repo --> 1 1 --> 2 2 --> 3 3 --> 4 4 --> 5 5 --> 6 6 --> result style 6 fill:#ff0000,stroke-width:4px,stroke:#000;
logscale
```
| sankey(source="Vendor.urlcategory", target="Vendor.domain")
```
Creates a Sankey diagram visualization where the source nodes are defined by Vendor.urlcategory and the target nodes are defined by Vendor.domain.
The width of the flows represents the frequency of access between each URL category and domain.
Event Result set.

Summary and Results

The widget is used to monitor and visualize the relationships between allowed URL categories and the specific domains accessed within those categories.

This widget is useful to understand how URL categorization policies are applied and identify patterns in allowed domain access across different categories.

Sample output from the incoming example data:

source	target	weight
News and Media	www.google.com	1
News and Media	www.microsoft.com	1
News and Media	www.nytimes.com	1
News and Media	www.youtube.com	1
News and Media	www.bing.com	1

The output shows the relationships between URL categories (source) and domains (target), with the connection strength indicated by the weight field.

Versions of this Page

Examples Library