ZPA

1. Zscaler Private Access
2. Architecture
   1. ZCC
      1. DNS Search Domains
   2. App connector
      1. Processes
   3. ZPA ZEN
   4. Dispatcher
   5. ZPA Private Service Edge (PSE)
      1. Disaster Recovery
3. Order of operation
   1. Access policy
   2. Isolation policy
   3. Timeout policy
   4. Certificates
   5. Browser Access
   6. AppProtection
   7. Double encryption
   8. VDI
   9. Machine Tunnel
   10. Privileged Remote Access (PRA)
4. Discovery
5. Peer-to-peer connectivity
   1. Server-to-client connectivity
   2. IP bindings
6. Log Streaming Service (LSS)
7. Deception

Zscaler Private Access

• no support
  • protocols, requiring ALG
  • bcast, mcast
  • ICMP
  • FTP active mode
  • UC
• timers
  • SCIM push period: commonly 40 mins
• SAML only
  • if IdP is not reachable via Internet
    1. enroll to ZPA from trusted network
    2. IdP – app segment
    3. authC timeout policy – never

Architecture

• app connector
• ZPA service edge: stitch µtunnels from user to app connector
  • public: own DC, AWS, Azure
  • private
• app segment: logical container
  • single app segment group
• server
  • mapping to several groups allowed (app connector, app segment)
  • dynamic discovery recommended
  • static server config
    • only subset of servers must be serving ZPA
    • ZPA users must be directed to IP other than resolved by DNS

ZCC

• first authC – to ZIA to get policies from Mobile admin
  • if ZIA tenant does not exist, dummy tenant is created
• µtunnel
  • 32-bit tag, globally unique
  • user data is cached before µtunnel is established
  • ZCC allows or rejects access to app based on policy
• forwarding modes: tunnel mode, none
  • Windows: WPF (no vNIC or RIB modification)
  • MAC: RIB + vNIC
  • Linux: RIB + vNIC
• automatic ZPA reauthC:
  • uses Windows (IWA) or MacOS credentials to authC with ZPA
  • IWA support must be enabled for IdP SSO domain

DNS Search Domains

• allows short names to be used instead of full FQDNs
• Domain Validation in Client Connector – if short name within domain is not found, return NXDOMAIN (otherwise search will continue through OS)

App connector

• in pairs
  • Linux: RPM (CentOS, Oracle, RHEL)
  • AWS, Azure
  • VM: ESXi, Hyper-V
  • single group
• 500 Mbps aggregate throughput
  • extra VMs if more throughput needed
• health checks for application
  • TCP: ports are open (handshake)
  • UDP: ICMP (empty UDP segment) or TCP connection on same port
  • FQDN: all IP addresses are reachable
  • IP: single IP is reachable
  • mode
    • continuous:
      • every 1s
      • next app and port are selected randomly
      • 10 ports per app
      • statically configured app only
    • on demand
      • reports during 30 mins after last access attempt
    • none
  • forwarded by Broker to every Dispatcher
• user IP address – app connector IP address (proxy)
  • does not accept inbound connections
• reverse proxy for the application

Processes

• zpa-connector
  • root
  • software upgrade
    • automatic process update
    • no OS update
    • update is signed with pinned certificate
  • child process restarts: automatic
• zpa-connector-child
  • zscaler user
  • hardware fingerprinting
    • provision_key.crypt – encrypted with hardware fingerprint
    • provision key is ignored if provision_key.crypt exists
    • includes MAC address → static MAC required
  • enrolls to ZPA
  • connects to apps and ZPA ZENs

ZPA ZEN

• broker
• terminates connection from users, app connectors and PSE: data and control
• co2br.prod.zpath.net: app connector to broker, DNS-based GSLB
• any.broker.prod.zpath.net: ZCC to broker, DNS-based GSLB
• enforces policy
• proxies control connections to other cloud components for users and app connectors
• full mesh to all Dispatchers
• sends list of ZPA applications to ZCC: domain, L4 protocol, port

Dispatcher

• no communication between Dispatchers
• app connector selection
  1. calculate RTT estimation
     • closer to user: between user and app connector group based on geolocation
     • closer to application: between app connector group and application
     • app connector group ≡ same geolocation
     • 1 ms per 160 km (2ms RTT)
  2. geofencing: local to country
     • outside China: China has least preference (last resort)
  3. threshold: select groups with RTT difference from best RTT less than 2ms
  4. user stickiness
  5. least utilization within the group
• instructs app connector to which ZPA ZEN to establish data connection (serving the user)
• aggregates application health data
  • DNS responses to ZCC are based on app reachability
  • DNS state (reachable or not) is cached

ZPA Private Service Edge (PSE)

• avoids backhauling to ZTE: deployed in DC or IaaS
  • connected to nearest public edge: mgmt and control traffic, log forwarding
    • connection to Dispatcher via public Broker
    • users still authenticate to public cloud
  • sends traffic to public edge for app available only through public edge
• single tenant
  • Publicly Accessible ≡ over the Internet
• VM, rpm package
  • 4h update window
  • one VM is updated in a group at a time
• 500 Mbps peak throughput
• requirements
  • in pairs
  • incoming TCP 443
  • unique IP address, static MAC address
  • certificate from same root CA as certificates for ZCC ID
  • SSL bypass for *.prod.zpath.net: no MITM (certificate pinning)
• selection criteria
  • closest geographically to ZCC
  • ZCC is on trusted network
• location policy: evaluated for new connections, no reevaluation
  • user has public IP: GeoIP
  • user has private IP: Service Edge location

Disaster Recovery

• protection against ZPA cloud failure
• modes:
  • test: runs alongside with ZPA cloud (DR drills)
  • on: actual DR event
  • off
• designated components:
  • ZPA Admin UI: app connector group + PSE group + app segments
    • locally cache subset of cloud config (every 15 minutes)
    • switching runtime modes forces reboot ⟹ sessions are disconnected
  • Mobile Admin UI: app profiles to enable users to access apps during DR
• DNS: DR domain
  • TXT record: customer initiated trigger of DR switchover
    • PSE, ZCC and app connectors poll for activated DR mode
      • missing record ≡ off mode
      • every 4 minutes
    • generated by DR tool
      • should be pre-generated and securely stored
      • only public key is stored by PSE, app connectors and ZCC
      • both activation and deactivation must be signed or unsigned
    • parameters: similar to DKIM
      • b: body (on, test, off)
      • bh: body hash
        • v;b;t;x;n must match for bh to be concatenated
        • signed SHA256(v;b;k;m;t;x)
        • base64 encoding
      • v: version
      • k: kind (zia, zpa, all)
      • m: ZIA forwarding mode
        • fo – send directly
        • fc – internet access disabled
        • wf – pre-selected destinations only
      • p: part index
      • n: number of parts
      • t: UTC issue time
      • x: UTC expiry time
    • DR domain and key pair cannot be changed during DR
  • A record: DNS round-robin over PSEs during DR
    • ZCC resolves records to PSE IPs in lieu of cloud LB
• DR operation
  • PSE and app connectors are disconnected from cloud
    • local logging only
    • no policies are applied
    • no new enrollments
    • no AppProtection, SIPA, WAF, Deception, LSS
  • ZCC only: Safe Mode in ZCC UI
    • Windows only
    • no Machine Tunnels
  • no SAML authC
    • client certificates must be valid: static issuer verification
      • no revokation check ⟹ certs must be purged via MDM
    • IdP is unreachable → SAML assertion validity is extended up to configured duration (start datetime + max age)

Order of operation

1. SAML attributes: user + device
2. SCIM attributes: user only
3. time policy
4. access policy
5. isolation policy
6. inspection policy
7. select app connector

Access policy

• first-match on hostname by default
  • if port is missing from specific entry but present in wildcard – drop traffic
• multimatch: several app segments can be matched on hostname
  • if port is missing from specific entry but present in wildcard – use wildcard
  • direction: from most specific to least specific
    • multimatch stops at the first segment with multimatch disabled
    • if no match is found before stop – use the entry with multimatch disabled, otherwise – exclude it from matched segments

Isolation policy

• HTTP(S) applications only

Timeout policy

• authC timeout
  • 7 days by default
  • new access requests only
• idle connection timeout
  • defaults
    • open TCP sessions: never
    • half-closed TCP session: 6 minutes
    • UDP session: 60 seconds

Certificates

• double-pinned: ZCC and app connector → no MITM
• pre-defined
  • client: root subordinate, ZCC ID certificates
  • root: self-signed, on tenant creation
  • connector: root subordinate, connector ID and server ID certificates
• custom certificate:
  • root CA
    • client and app connector certificates – via CSR
  • double encryption
• delete certificate ≡ revoke relevant certificates

Browser Access

• pixel streaming: BYOD, Day1 acquisitions, contractors, Chromebooks
  • container – per user per browser
    • Chromium
    • 10 minutes timeout
• ZCC has precedence over BA: BA relies on DNS resolution, ZCC intercepts DNS requests
  • can be overcome by adjusting hosts file
• exporter ≡ ZCC alternative within ZPA cloud, reverse proxy
  • client type = Cloud Browser in Access Policy
  • certificate per single (FQDN cert) or several (wildcard cert) app segments
    • wildcard cert matches only one level
    • wildcard in ZPA matches all levels
  • use distributed storage for session info
  • client identification is based on SNI – domains must be unique
• HTTPS applications only (HTTP is redirected to HTTPS)
  • CNAME is used to redirect application to BA exporter (app segment config)
  • DNS-based GSLB
  • ultimate IPs – LBs, not reachable via ICMP (curl for testing)
  • authC cookies:
    • ZPA cloud does not have access to SAML assertion directly at rest
    • per user per browser
    • encryption cookie: key to decrypt SAML assertion, generated and installed by exporter during redirection to IdP
    • session cookie: encrypted SAML assertion
• CORS:
  • ZPA does not return CORS headers by default → JS is not allowed to load resources from other domains by browser
  • JS is not allowed to load resources from another domain using the same authC cookies → withCredentials: true in JS application allows using cookies from other domains
    • has to be restricted by allowed list of domains (Access-Control-Allow-Origin)
    • vulnerable to CSRF within the ACL
• double encryption not supported
• user portal
  • published by Zscaler
  • tiles, available to user – according to Access policy
• different internal and external FQDNs (including internal IPs)
  • dynamic server discovery off
    • static server IPs are enumerated
    • multiple web servers
    • no LB needed
  • dynamic server discovery on
    • resolution is based on external FQDN → internal DNS must define A record for external FQDN
    • single web server
    • LB required

AppProtection

• WAF alternative
  • OWASP top 10 vulnerabilities scanning
  • HTTP header inspection and custom rules
• after access control policies

Double encryption

• for unencrypted protocols
• 2x performance decrease: 250 Mbps throughput of app connector
• TLS tunnel between app connector and ZCC → encrypted traffic on ZEN
• enabled per FQDN, port is ignored
• requires custom certificate
  • ZCC and app connector require re-provisioning

VDI

• dedicated, single-user only
• must not log in on master VM
• recommendation
  • strict enforcement enabled
  • hide app UI on launch disabled

Machine Tunnel

• Windows only
• pre-login: connectivity to AD
  • –policyToken: App Profile ID for machines before user enrolls
  • Machine Authentication Required: user logs in with IdP before machine tunnel can be built
• device authC: machine provisioning key
  • signed by CA used for client certificate signing
  • replaced on user login
  • rebuilt on logout

Privileged Remote Access (PRA)

• SSH and RDP proxy

Discovery

• exclude DNS from discovery: tunnel DNS ≡ resolve to internal IP, not synthetic
• configured applications are not included in wildcard because of health check (includes port)
  • can be still reached if health check is None
• DNS SRV record might bypass access policy through wildcard application
  • can be matched explicitly by using port TCP 1

Peer-to-peer connectivity

• Client Hostname Validation – regex to select devices with ZCC for peer-to-peer communication
  • unique FQDNs
  • requires DNS suffix to be correctly configured on hosts
    • domain join
      • Windows, Linux: recommended
      • MacOS: mandatory
  • ZCC must be allowed to collect hostname info (MobileAdmin portal)
• app segments with client FQDNs govern connectivity
  • server group is not required
  • by default permit all within the group of hosts, configured for peer-to-peer access
• active ZCC or machine tunnel are required
  • access from ZCC to interface IPs – from 127.0.0.1 (must be allowed on FW)

Server-to-client connectivity

• Branch or Cloud connectors are required instead of ZCC
  • gateway for Zscaler IP range

IP bindings

• use for IP-based connection in lieu of FQDN-based
• non-overlapping subnets
  • virtual IP is assigned to client device with ZCC
  • IPs are not configured on NICs
  • if pool overflows, IPs from other pools are allocated
• not supported
  • DR
  • machine tunnels
  • µ-tenants
  • multiple tenants

Log Streaming Service (LSS)

• ZPA only
• AppConnectors function
  • dedicated AppConnectors if log volume is high
• receives compressed events from Nanolog and forwards to SIEM

Deception

• honeypot
• decoys: container or VM (Windows)
  • file
  • process: AV decoys for malware to target
  • AD server: account decoys
  • DNS reconaissance: Threat Intelligence decoys
  • network scanning
• decoys are provided by ZCC, decoy connector or ZPA
• decoy connector VM
  • interfaces:
    • mgmt
    • decoy: trunk
  • IP pool for decoys
  • SSL: pinned certificate, connection to admin UI
• actions: ZPA policy or 3rd party (e.g., EDM)
• no ZPA wildcard app segments (not a real application but an umbrella)