This is a small example on how to configure policy based forwarding (PBF) on a Palo Alto Networks firewall. The use case was to route all user generated http and https traffic through a cheap ADSL connection while all other business traffic is routed as normal through the better SDSL connection. Since I ran into two problems with this simple scenario, I am showing the solutions here.

The covered PAN-OS version during this tests was 6.0.0.

The mere setup is really easy: The SDSL router is behind eth1/1 and a default route on the PA points to that router. The ADSL router resides behind eth1/2 and has NO static route entry in the router on the PA. No second virtual router or the like is needed. (This limits the usage of the second ISP connection. It can only be used for this policy based forwarding and not for incoming connections. E.g., no remote access VPN tunnels can terminate at this second connection since it has no default route to the router backwards. However, static Site-to-Site VPNs could be used if the remote endpoint has an entry in the routing table.)

The routing decision based on the destination ports 80 and 443 are made within the Policy Based Forwarding rules in the Policies tab. The following screenshots document my policy. Note that I have NOT selected the applications “web-browsing” and “ssl” but the mere ports, i.e., services. This is due to the fact that the PA cannot decide which application it sees based on the very first packet. Therefore, I simply forward all requests to the ports 80 and 443 to the ADSL connection:

The following screenshot shows that the same destination is called once with ping and once with http. According to the PBF, both connections take a different egress interface:

Do not PBF my Private Networks

My policy was a bit too wide: I was not able to reach my own http servers on the LAN anymore. Of course, the single PBF rule forwards all http requests to the ADSL router. The solution was to add a second PBF rule BEFORE the already existing one, which has the destination IP addresses set to all the internal IPv4 addresses (e.g., all RFC1918 addresses) and an action of “No PBF”.

IPv4 to the Left, IPv6 to the Right

Another problem in my scenario was IPv6 since I could not route my global unicast IPv6 space from the ISP with the SDSL connection through another ADSL connection. (Really bad, because IPv6 is the solution for so many other cases. Here it is a bit difficult. And yes, the hated NAT for IPv4 makes it easy to use PBF in this scenario.)

The solution was to configure another “no-pbf” rule that forwards all IPv6 packets to its normal default router which is of course capable of this global unicast IPv6 range.

Here is a screenshot of my final policy:

After that, all connections worked as expected. To show both links at the same time, a homepage that reveals both Internet protocol addresses such as the german www.wieistmeineip.de site can be used. Here, the IPv4 address from the dynamic ADSL connection as well as the global unicast IPv6 address (with privacy extensions enabled) are shown:

The egress interface can be seen in the traffic log. IPv4 connections are correctly forwarded to the ADSL router on eth1/2 while IPv6 traffic still goes out on eth1/1:

Reference

• Palo Alto: Policy Based Forwarding

This is a tutorial on how to configure the GlobalProtect Gateway on a Palo Alto firewall in order to connect to it from a Linux computer with vpnc.

Short version: Enable IPsec and X-Auth on the Gateway and define a Group Name and Group Password. With this two values (and the gateway address), add a new VPN profile within vpnc on the Linux machine. Login with the already existing credentials.

Long version with screenshots comes here:

I assume that an already working GlobalProtect configuration is in place. The tested PAN-OS version was 6.0.1.

Configuration Palo Alto

The main step is the activation of IPsec (which is useful for the mere GlobalProtect client, too), and the X-Auth Support on the GlobalProtect Gateway. A group name and group password must be set, just like the VPN-Client settings on a Cisco ASA firewall.

Furthermore, the “from untrust to untrust” security policy must be expanded with at least the application “ciscovpn“.  But due to the application dependency warnings after a successful commit on the PA, it is less annoying if “dtls” and all the other dependencies for ciscovpn are allowed, too, though they are not needed. In this way, the commit warnings can be reduced.

That is, I am permitting the following applications for the complete GlobalProtect process, incl. GlobalProtect client, etc.:

Linux: vpnc

I ran a Ubuntu 13.10 with Linux kernel 3.11.0-18 on my test machine.

The following two applications must be installed:

sudo apt-get install vpnc network-manager-vpnc

To add a VPN connection, click on the network symbol in the upper right corner: VPN-Connections -> VPN configuration -> Add -> Cisco VPN-Client (vpnc). Give it a name and fill in the gateway name/address, the username and the groupname & -password of the just configured GlobalProtect Gateway (sorry for the German screenshot):

Test

To connect to the VPN endpoint, click on the new VPN profile and type in your account name and password. After a few seconds, the VPN tunnel should be established.

Here are two listings of the IP address of my Linux test machine (

ip a s

ip r s

weberjoh@JW-NB01-Ubuntu:~$ ip a s
1: lo: <loopback,up,lower_up> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <no-carrier,broadcast,multicast,up> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
    link/ether 00:15:c5:16:6a:d2 brd ff:ff:ff:ff:ff:ff
3: wlan0: <broadcast,multicast,up,lower_up> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:13:02:47:49:37 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.36/24 brd 192.168.1.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::213:2ff:fe47:4937/64 scope link 
       valid_lft forever preferred_lft forever

weberjoh@JW-NB01-Ubuntu:~$ ip r s
default via 192.168.1.1 dev wlan0  proto static 
192.168.1.0/24 dev wlan0  proto kernel  scope link  src 192.168.1.36  metric 9

While the following shows the values within the VPN tunnel. A new tun0 interface is present and the default route points to that tun0 interface:

weberjoh@JW-NB01-Ubuntu:~$ ip a s
1: lo: <loopback,up,lower_up> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <no-carrier,broadcast,multicast,up> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
    link/ether 00:15:c5:16:6a:d2 brd ff:ff:ff:ff:ff:ff
3: wlan0: <broadcast,multicast,up,lower_up> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:13:02:47:49:37 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.36/24 brd 192.168.1.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::213:2ff:fe47:4937/64 scope link 
       valid_lft forever preferred_lft forever
4: tun0: <pointopoint,multicast,noarp,up,lower_up> mtu 1412 qdisc pfifo_fast state UNKNOWN qlen 500
    link/none 
    inet 192.168.126.1/32 brd 192.168.126.1 scope global tun0
       valid_lft forever preferred_lft forever

weberjoh@JW-NB01-Ubuntu:~$ ip r s
default dev tun0  proto static 
80.154.108.228 via 192.168.1.1 dev wlan0  proto static 
192.168.1.0/24 dev wlan0  proto kernel  scope link  src 192.168.1.36  metric 9

And by the way: the DNS server in /etc/resolv.conf is NOT changed during the VPN connection. Only the search domain (DNS suffix) correspondent to the network settings in the GlobalProtect Gateway is appended.

Here are some screenshots of the Palo Alto firewall: The first one shows the Gateway Remote Users with a client of “Linux…”, while the second screenshot shows the System Log with detailed information about the GlobalProtect session: It is recognized as a Cisco VPN Client. After the finished session, the Traffic Log shows at least two sessions with “ciscovpn”, one on port 500 (IKE) and one on port 4500 (ESP inside UDP).

And as always, I am using my http://ip.webernetz.net script to show my current Internet IP address which reveals in this case, that I am surfing through the Palo Alto ISP connection.

Links

• Similar blog post: Connecting to a Palo Alto Network GlobalProtect Gateway from Linux
• Palo Alto: Split Tunneling for VPNC Client on Linux Distributions

I tested the Palo Alto GlobalProtect app on my iPhone, but also the native IPsec Cisco VPN-Client on iOS which connects to the GlobalProtect Gateway on a Palo Alto firewall, too. Since this variant needs no further licenses from Palo Alto, it is a cheap alternative for a basic VPN connection.

Though not that much exciting, there are a few differences in the logs on the firewall which I will show here on the basis of a few screenshots.

During this tests I ran PAN-OS 6.0.1 with a GlobalProtect Agent of 2.0.1. My iPhone ran iOS version 7.0.6.

The GlobalProtect app from Palo Alto works without any problems if a correct Portal and Gateway are already configured. In order to use the native Cisco IPsec client on iOS, the “X-Auth Support” must be enabled on the GlobalProtect Gateway, such as shown here in my post about the Linux vpnc client.

GlobalProtect vs. iOS IPsec Client

In order to have the best performance and configuration possibilities, the GlobalProtect app from Palo Alto should be used. With this app, options such as HIP profiles can be used, the best Gateway can be determined after a connect to the Portal, etc. However, it requires the GlobalProtect Gateway license [reference].

So, if only the mere VPN connection to the internal network is needed, the built-in IPsec VPN-Client from Cisco fits completely. It uses the IKE and ESP protocols (instead of SSL) to connect to the Gateway firewall.

Here are two screenshots of both variants from the iPhone. Since the configuration steps are really simple (only the gateway, groupname & -password are needed), I am not showing them here in detail.

PA: Remote Users Log

Here are the log differences on the Gateway -> Remote Users information page. If a user logs in with the GlobalProtect app, the client is recognized as “Apple iOS …”, while a native IPsec connections is shown as “iPhone OS: …”:

PA: System Log

The System Logs under the Monitor tab are a bit more different. Here is the GlobalProtect log with much more lines compared to the IPsec logs beneath. This is because the app first connects to the Portal and then to the Gateway:

These are the native IPsec System logs with a direct connection to the Gateway:

Further Reading

• Palo Alto: GlobalProtect Licenses
• Palo Alto: GlobalProtect Configuration Tech Note
• Palo Alto: GlobalProtect Configuration for the IPsec Client on Apple iOS Devices

When talking about VPNs it is almost always clear that they are encrypted. However, it is not so clear on which security level a VPN is established. Since the Perfect Forward Secrecy (PFS) values of “DH group 5″ etc. do not clearly specify the “bits of security”, it is a misleadingly assumption that the security is 256 bits due to the symmetric AES-256 cipher. It is not! Diffie-Hellman group 5 has only about 89 bits of security…

Therefore, common firewalls implement DH group 14 which has a least a security level of approximately 103 bits. I tested such a site-to-site VPN tunnel between a Palo Alto and a Juniper ScreenOS firewall which worked without any problems.

Bits of Security

The traffic over a VPN is encrypted with a symmetric cipher such as AES, but the encryption key is generated with an asymmetric cipher such as Diffie-Hellman. An attacker who captures the complete traffic of a VPN might be able to brute-force the used keys of this Diffie-Hellman key exchange OR he could do a brute-force attack of the encrypted traffic with AES. Of course he would do that attack on the weaker cipher, i.e., on DH and not on AES.

On this Crypto++ wiki, a table that lists the bits of security for different symmetric and asymmetric functions can be found, taken from this PDF file from NIST on p. 64. Another resource is the ECRYPT2 Yearly Report on Algorithms and Keysizes (2010) which has the following (lower) values on p. 30:

• DH with 1024 bits (group 2) has 73 bits of security
• DH with 1536 bits (group 5) has 89 bits of security
• DH with 2048 bits (group 14) has 103 bits of security

That is: If a really secure VPN connection is needed, the phase 1 and phase 2 parameters should use at least Diffie-Hellman group 14 to gain 103 bits of security. Furthermore, at least AES-128 can be used, which has a security of almost 128 bits. However, since AES-256 can be used without any troubles, I don’t know why AES-128 should be used instead (except high CPU load issues).

Test VPN Palo <-> Juniper

While I expect that such VPN settings between firewalls of the same vendor work without any problems, I configured DH group 14 with AES-256 and SHA-256 (also new, instead of SHA-1) for both IKE and IPsec (ESP) on my test VPN between a Palo Alto PA-200 (6.0.1) and a Juniper SSG 5 (6.3.0r16a.0) firewall. It worked.

Here is an appropriate listing from the Palo Alto firewall that shows these correct values for the VPN (DH14/…):

weberjoh@fd-wv-fw02> show vpn ike-sa gateway fd-wv-fw01

phase-1 SAs
GwID/client IP  Peer-Address           Gateway Name           Role Mode Algorithm          Established     Expiration      V  ST Xt Phase2
--------------- ------------           ------------           ---- ---- ---------          -----------     ----------      -  -- -- ------
              1 172.16.1.1             fd-wv-fw01             Init Main PSK/DH14/A256/SHA256 Mar.27 12:38:57 Mar.27 20:38:57 v1 12  4      1

Show IKEv1 IKE SA: Total 1 gateways found. 1 ike sa found.

phase-2 SAs
GwID/client IP  Peer-Address           Gateway Name           Role Algorithm               SPI(in)  SPI(out) MsgID    ST Xt
--------------- ------------           ------------           ---- ---------               -------  -------- -----    -- --
              1 172.16.1.1             fd-wv-fw01             Init DH14/tunl/ESP/A256/SHA256 A2C9F430 38921FFB D8170C56  9  1

Show IKEv1 phase2 SA: Total 1 gateways found. 1 ike sa found.

And similarly, two Juniper listings (grp14/…):

fd-wv-fw01-> get ike cookies

80102f/0003, 172.16.1.2:500->172.16.1.1:500, PRESHR/grp14/AES256/SHA2-256, xchg(5) (fd-wv-fw02/grp-1/usr-1)
resent-tmr 322 lifetime 28800 lt-recv 28800 nxt_rekey 21359 cert-expire 0
responder, err cnt 0, send dir 1, cond 0x0
nat-traversal map not available
ike heartbeat              : disabled
ike heartbeat last rcv time: 0
ike heartbeat last snd time: 0
XAUTH status: 0
DPD seq local 0, peer 2491

fd-wv-fw01-> get sa id 00000002
index 0, name fd-wv-fw02, peer gateway ip 172.16.1.2. vsys
auto key. tunnel if binding node, tunnel mode, policy id in:<-1> out:<-1> vpngrp:<-1>. sa_list_nxt:<-1>.
tunnel id 2, peer id 0, NSRP Local.     site-to-site. Local interface is ethernet0/6 <172.16.1.1>.
  esp, group 14, a256 encryption, s256 authentication
  autokey, IN active, OUT active
  monitor<1>, latency: 2, availability: 100
  DF bit: clear
  app_sa_flags: 0x24001a7
  proxy id: local 0.0.0.0/0.0.0.0, remote 0.0.0.0/0.0.0.0, proto 0, port 0/0
  ike activity timestamp: 1987333985
  DSCP-mark : disabled
nat-traversal map not available
incoming: SPI 38922007, flag 00004000, tunnel info 40000002, pipeline
  life 3600 sec, 3397 remain, 0 kb, 0 bytes remain
  anti-replay on, last 0x47, window 0xffffffff, idle timeout value <0>, idled 1 seconds
  next pak sequence number: 0x0
  bytes/paks:51371280/706390; sw bytes/paks:51371280/706390
outgoing: SPI fd1321fd, flag 00000000, tunnel info 40000002, pipeline
  life 3600 sec, 3397 remain, 0 kb, 0 bytes remain
  anti-replay on, last 0x0, window 0x0, idle timeout value <0>, idled 1 seconds
  next pak sequence number: 0x47
  bytes/paks:51371600/706400; sw bytes/paks:51371600/706400

I also tried to configure group 14 values on my VPNs to a Cisco ASA 5505 (9.1(4)) firewall, but an error message said that DH14 is only supported on IKEv2. Hm. Maybe the two other vendors implemented it on their own, or Cisco did not want to implement it for IKEv1. I don’t know.

Further Reading

• At a Glance: Perfect Forward Secrecy
• RFC 3526: More Modular Exponential (MODP) Diffie-Hellman groups for Internet Key Exchange (IKE)
• RFC 5114: Additional Diffie-Hellman Groups for Use with IETF Standards

This time I configured a static S2S VPN between a Palo Alto firewall and a Cisco IOS router. Here comes the tutorial:

I am not using a virtual interface (VTI) on the Cisco router in this scenario, but the classical policy-based VPN solution. That is, no route entry is needed on the Cisco machine. However, the Palo Alto implements all VPNs with tunnel interfaces. Hence, a route to the tunnel and Proxy IDs must be configured. (I also wrote a guide for a route-based VPN between a Cisco router and a Palo Alto firewall here.)

In my test lab I am running a PA-200 with PAN-OS 6.0.3. The Cisco router is an old Cisco 2621 with IOS 12.3(26) and image “c2600-ik9o3s3-mz.123-26.bin”.

Laboratory

The following figure depicts my test laboratory:

Palo Alto

The configuration steps for the Palo Alto Networks firewall are the following:

1. IKE and IPSec Crypto profiles, e.g., aes256, sha1, pfs group 5, lifetime 8h/1h.
2. IKE Gateway with the pre-shared key and the corresponding IKE Crypto Profile. The “Identification” fields are not needed.
3. Tunnel Interface within a virtual router (e.g., “default”) and a security zone (e.g., “vpn-s2s”). The interface does not need an IP address.
4. IPSec Tunnel: Tying all together: tunnel interface, IKE gateway, IPSec crypto profile. Furthermore, the Proxy IDs (= protected networks) are set here.
5. Static route to the destination network through the tunnel interface (without next hop address).
6. Policy from untrust to untrust with the applications “ike”, “ipsec”, and “ciscovpn” allowed.
7. Policies from trust zones to the zone in which the tunnel interface resides.

Here are my configuration screenshots:

Cisco Router

The Cisco router, configured through the CLI, needs the following lines:

1. crypto isakmp appropriate to the “IKE Crypto” on the PA
2. crypto isakmp key with the pre-shared key
3. crypto ipsec appriopriate to the “IPSec Crypto” on the PA
4. access-list which defines the protected networks, corresponding to the “Proxy IDs”
5. crypto map with the transform-set, peer, pfs group, and access-list
6. crypto map applied to the outside interface
7. (Note: No route entry is needed since this VPN is a policy-based VPN which makes its routing decision based on the access-list.)

Here is the bunch of my configuration commands:

!
crypto isakmp policy 10
 encr aes 256
 authentication pre-share
 group 5
 lifetime 28800
!
crypto isakmp key 3WU99OhGqpPRIhjNm2eBeBG6hQbLQ5 address 172.16.1.2
!
crypto ipsec transform-set aes256-sha esp-aes 256 esp-sha-hmac
!
crypto map map01 1 ipsec-isakmp
 set peer 172.16.1.2
 set transform-set aes256-sha
 set pfs group5
 match address acl-vpn-PA
!
interface FastEthernet0/0
 crypto map map01
!
ip access-list extended acl-vpn-PA
 permit ip 192.168.141.0 0.0.0.255 192.168.121.0 0.0.0.255
!

Monitoring

After a successful establishment of the tunnel, the PA shows green bubbles in its IPSec Tunnels overview. The Session Browser reveals active sessions for ike or ciscovpn and ipsec-esp. However, I noticed that after these sessions are gone, only the ike sessions are in my traffic log, while the ipsec sessions are not correct according to the listed traffic bytes. Hm. Has anyone else recognized a similar behaviour?

The Cisco router can be queried with the subsequent commands:

• show crypto isakmp sa detail
• show crypto ipsec sa
• show crypto map

fd-wv-ro02#show crypto isakmp sa detail
Codes: C - IKE configuration mode, D - Dead Peer Detection
       K - Keepalives, N - NAT-traversal
       X - IKE Extended Authentication
       psk - Preshared key, rsig - RSA signature
       renc - RSA encryption

C-id  Local           Remote          I-VRF    Encr Hash Auth DH Lifetime Cap.
509   172.16.1.4      172.16.1.2               aes  sha  psk  5  01:48:09

-------------------------------------------------------------------------------

fd-wv-ro02#show crypto ipsec sa

interface: FastEthernet0/0
    Crypto map tag: map01, local addr. 172.16.1.4

   protected vrf:
   local  ident (addr/mask/prot/port): (192.168.141.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (192.168.121.0/255.255.255.0/0/0)
   current_peer: 172.16.1.2:500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 42122, #pkts encrypt: 42122, #pkts digest 42122
    #pkts decaps: 47189, #pkts decrypt: 47189, #pkts verify 47189
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 2, #recv errors 0

     local crypto endpt.: 172.16.1.4, remote crypto endpt.: 172.16.1.2
     path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet0/0
     current outbound spi: C0F1609E

     inbound esp sas:
      spi: 0x81DF870F(2178909967)
        transform: esp-256-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        slot: 0, conn id: 2080, flow_id: 81, crypto map: map01
        sa timing: remaining key lifetime (k/sec): (4516253/1975)
        IV size: 16 bytes
        replay detection support: Y

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xC0F1609E(3237044382)
        transform: esp-256-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        slot: 0, conn id: 2081, flow_id: 82, crypto map: map01
        sa timing: remaining key lifetime (k/sec): (4516252/1973)
        IV size: 16 bytes
        replay detection support: Y

     outbound ah sas:

     outbound pcp sas:

-------------------------------------------------------------------------------

fd-wv-ro02#show crypto map
Crypto Map "map01" 1 ipsec-isakmp
        Peer = 172.16.1.2
        Extended IP access list acl-vpn-PA
            access-list acl-vpn-PA permit ip 192.168.141.0 0.0.0.255 192.168.121.0 0.0.0.255
        Current peer: 172.16.1.2
        Security association lifetime: 4608000 kilobytes/3600 seconds
        PFS (Y/N): Y
        DH group:  group5
        Transform sets={
                aes256-sha,
        }

        Interfaces using crypto map map01:
                FastEthernet0/0

And one more time: Since the Cisco Router decides its forwarding decisions for VPNs on the policy (ACL) and NOT on route entries, the routing table does NOT show any of my site-to-site remote networks, but only the connected and static configured routes:

fd-wv-ro02#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 172.16.1.1 to network 0.0.0.0

S    192.168.120.0/24 [1/0] via 172.16.1.2
S    192.168.125.0/24 [1/0] via 172.16.1.2
     172.16.0.0/24 is subnetted, 1 subnets
C       172.16.1.0 is directly connected, FastEthernet0/0
C    192.168.140.0/24 is directly connected, FastEthernet0/1.140
C    192.168.141.0/24 is directly connected, FastEthernet0/1.141
S*   0.0.0.0/0 [1/0] via 172.16.1.1

Links

Similar information about this tunnel can be found here:

• Creating a Site to Site IPSec VPN with a Palo Alto Networks Application Firewall and a Cisco Router
• [Video] Cisco to Palo Alto IPsec Tunnel: The perfect couple?

One more VPN article. Even one more between a Palo Alto firewall and a Cisco router. But this time I am using a virtual tunnel interface (VTI) on the Cisco router which makes the whole VPN set a “route-based VPN”. That is: Both devices decide their traffic flow merely based on the routing table and not on access-list entries. In my opinion, this is the best way to build VPNs, because there is a single instance (the routing table) on which a network admin must rely on in order to investigate the traffic flow.

Note that I also wrote a blog post about the “policy-based VPN” between a Cisco router and the Palo Alto firewall. This here is mostly the same on the Palo Alto side while some other commands are issued on the Cisco router.

My lab units are a Palo Alto PA-200 with PAN-OS 6.0.3 and a Cisco router 2811 with IOS 12.4(24)T8 (c2800nm-advipservicesk9-mz.124-24.T8.bin).

Laboratory

Palo Alto

These are the configuration steps on the Palo Alto firewall:

1. IKE and IPSec Crypto profiles, e.g., aes256, sha1, pfs group 14 (!), lifetime 8h/1h.
2. IKE Gateway with the pre-shared key and the corresponding IKE Crypto Profile. The “Identification” fields are not needed.
3. Tunnel Interface with an IPv4 address within a virtual router (e.g., “default”) and a security zone (e.g., “vpn-s2s”).
4. IPSec Tunnel: Tying all together: tunnel interface, IKE gateway, IPSec crypto profile. The Tunnel Monitor can be used to ping the other side of the tunnel. And since this is a route-based VPN, the Proxy IDs are not needed here!
5. Static route to the destination network through the tunnel interface with a next-hop address of the tunnel interface IP address of the other side.
6. Policy from untrust to untrust with the applications “ike”, “ipsec”, and “ciscovpn” allowed.
7. Policies from trust zones to the zone in which the tunnel interface resides, and vice versa.

Here are my test lab screenshots:

Cisco Router

The Cisco router, configured through the CLI, needs the following lines:

1. crypto isakmp policy appropriate to the “IKE Crypto” on the PA
2. crypto isakmp key with the pre-shared key
3. crypto ipsec transform-set appropriate to the “IPSec Crypto” on the PA
4. crypto ipsec profile that points to the transform-set and has pfs group14 set
5. interface Tunnel with an IPv4 address, tunnel source and destination addresses (outside addresses of the router and the Palo Alto), tunnel mode of ipsec and a reference to the crypto profile
6. Finally, a static ip route through the tunnel interface to the tunnel IPv4 address of the Palo Alto side
7. (Note, there is no “crypto map” used here, the real outside interface is not touched, and no access-list is configured!)

Here is the bunch of my configuration commands:

crypto isakmp policy 10
 encr aes 256
 authentication pre-share
 group 14
 lifetime 28800
!
crypto isakmp key 5Tc56Q5A4HcLB72AUQqhzAFcXY0Lm8 address 172.16.1.2
!
crypto ipsec transform-set aes256-sha esp-aes 256 esp-sha-hmac
!
crypto ipsec profile PA
 set transform-set aes256-sha
 set pfs group14
!
interface Tunnel121
 ip address 10.0.0.6 255.255.255.252
 tunnel source 172.16.1.5
 tunnel destination 172.16.1.2
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile PA
!
ip route 192.168.121.0 255.255.255.0 Tunnel121 10.0.0.5

Don’t forget the Route

During the first test in my lab, the VPN was up but the traffic flew through the network even without the route through the tunnel! This was because the normal routing could reach the network from the Cisco router to the Palo Alto directly. So, one more time: Check the routing through the correct VPN tunnel! (This becomes even more serious when IPv6 with global unicast addresses will be used, in which situation every address could reach every other address on the Internet even without VPNs.)

The following screenshot shows two traceroute commands on a Linux test machine. During the first command, the route entry on the Cisco router was not added (though the VPN tunnel was already established), while the second traceroute went over the correct VPN tunnel:

Monitoring

Green bubbles in the Palo Alto tunnel pane and system log entries corresponding to the IKE gateway and IPsec tunnel objects. (Note the hourly refresh of the key due to the IPsec lifetime of 3600 sec):

Show commands on the Cisco router:

fd-wv-ro03#show crypto isakmp sa detail
Codes: C - IKE configuration mode, D - Dead Peer Detection
       K - Keepalives, N - NAT-traversal
       T - cTCP encapsulation, X - IKE Extended Authentication
       psk - Preshared key, rsig - RSA signature
       renc - RSA encryption
IPv4 Crypto ISAKMP SA

C-id  Local           Remote          I-VRF    Status Encr Hash Auth DH Lifetime Cap.

1135  172.16.1.5      172.16.1.2               ACTIVE aes  sha  psk  14 05:25:04
       Engine-id:Conn-id =  SW:135

IPv6 Crypto ISAKMP SA

---------------------------------------

fd-wv-ro03#show crypto ipsec sa

interface: Tunnel121
    Crypto map tag: Tunnel121-head-0, local addr 172.16.1.5

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 172.16.1.2 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 679743, #pkts encrypt: 679743, #pkts digest: 679743
    #pkts decaps: 680929, #pkts decrypt: 680929, #pkts verify: 680929
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 66, #recv errors 0

     local crypto endpt.: 172.16.1.5, remote crypto endpt.: 172.16.1.2
     path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet0/0
     current outbound spi: 0xFF5F2879(4284426361)
     PFS (Y/N): Y, DH group: group14

     inbound esp sas:
      spi: 0xFA8A1BF5(4203355125)
        transform: esp-256-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 4161, flow_id: NETGX:2161, sibling_flags 80000046, crypto map: Tunnel121-head-0
        sa timing: remaining key lifetime (k/sec): (4436876/1194)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xFF5F2879(4284426361)
        transform: esp-256-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 4162, flow_id: NETGX:2162, sibling_flags 80000046, crypto map: Tunnel121-head-0
        sa timing: remaining key lifetime (k/sec): (4436876/1194)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

---------------------------------------

fd-wv-ro03#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 172.16.1.1 to network 0.0.0.0

S    192.168.121.0/24 [1/0] via 10.0.0.5, Tunnel121
C    192.168.151.0/24 is directly connected, FastEthernet0/1.151
S    192.168.120.0/24 [1/0] via 172.16.1.2
C    192.168.150.0/24 is directly connected, FastEthernet0/1.150
S    192.168.111.0/24 [1/0] via 10.0.0.9, Tunnel111
S    192.168.125.0/24 [1/0] via 172.16.1.2
     172.16.0.0/24 is subnetted, 1 subnets
C       172.16.1.0 is directly connected, FastEthernet0/0
     10.0.0.0/30 is subnetted, 2 subnets
C       10.0.0.8 is directly connected, Tunnel111
C       10.0.0.4 is directly connected, Tunnel121
S*   0.0.0.0/0 [1/0] via 172.16.1.1

Done

Links

• Cisco: Configuring a Virtual Tunnel Interface with IP Security
• Cisco: IPSec Virtual Tunnel Interface
• Palo Alto Networks: Site-to-Site IPSec VPN Between Palo Alto Networks Firewall and Cisco Router using VTI Not Passing Traffic

It was not easy for me to understand the type of zones and “from – to” policy definitions when working with a Palo Alto firewall that has multiple vsys’s and shared gateways. I was missing an at-a-glance picture that shows which zones to use. (Though this document describes the whole process quite good.) So, here comes one…

Not much to say about the figure. The security zones are dark blue. The colored arrows show sample policy definitions, while the boxes outside describe the traffic log entries.

I hope that this figure increases the understanding of the inter-vsys setup. If not, please write a comment.

Links

• Palo Alto: How to setup Shared Gateway and Inter VSYS Setup in PAN-OS 4.0
• Palo Alto: VSYS Shared Gateways

I tested OSPF for IPv4 in my lab: I configured OSPF inside a single broadcast domain with five devices: 2x Cisco Router, Cisco ASA, Juniper SSG, and Palo Alto PA. It works perfectly though these are a few different vendors.

I will show my lab and will list all the configuration commands/screenshots I used on the devices. I won’t go into detail but maybe these listings help for a basic understanding of the OSPF processes on these devices.

I don’t want to say much about OSPF. Whoever reaches this post might already know about it. (Or read the articles about OSPF on Wikipedia or Cisco.)

Lab

This figure shows my lab and the basic OSPF values:

Note that I have a few more networks and Site-to-Site VPNs between these devices. So this figure is not complete at all but shows all relevant OSPF objects.

Some information

• Everything is in area 0.0.0.0, type broadcast
• Juniper SSG should be the DR: interface priority set to 100.
• Palo Alto PA should be the BDR: interface priority set to 50.
• Router-ID is always set manually to the IPv4 address of the interface (172.16.1.x).
• Cost for the interfaces as seen in the figure. For the Cisco routers I used the 

  auto-cost reference-bandwidth 10000

   command, while for all the other devices I configured them manually.
• Passive-interface on all user/access interfaces.
• Static routes are redistributed on a few devices for Remote Access VPN (Cisco ASA, Palo Alto) and Site-to-Site VPNs (Juniper).
• The default route to the Internet via the Juniper SSG is also redistributed. It has a cost of 42 because it is the answer of everything.
• No changes in the administrative distance / route preference, though it is different on all devices (Cisco: 110, Juniper: 60, Palo Alto: 30). However, since I am only using one dynamic routing protocol, this does not matter since it is only for local relevance on each firewall.

Of course, these are only the basic configurations for OSPF. I have not worked with authentication between the neighbors, nor have I fine-tuned other parameters such as graceful restart (non-stop forwarding), etc.

Cisco Router

I have two Cisco routers in my lab: One 2621 with IOS version 12.3(26) and one 2811 with IOS 12.4(24)T8.

This is the configuration for one of the Cisco routers. The config of the other router looks exactly the same:

router ospf 1
 router-id 172.16.1.5
 log-adjacency-changes
 auto-cost reference-bandwidth 10000
 passive-interface default
 no passive-interface FastEthernet0/0
 network 172.16.1.0 0.0.0.255 area 0.0.0.0
 network 192.168.150.0 0.0.0.255 area 0.0.0.0
 network 192.168.151.0 0.0.0.255 area 0.0.0.0

And here are two show commands:

fd-wv-ro03#show ip ospf neighbor

Neighbor ID     Pri   State           Dead Time   Address         Interface
172.16.1.1      100   FULL/DROTHER    00:00:33    172.16.1.1      FastEthernet0/0
172.16.1.2       50   FULL/BDR        00:00:37    172.16.1.2      FastEthernet0/0
172.16.1.3        1   FULL/DROTHER    00:00:36    172.16.1.3      FastEthernet0/0
172.16.1.4        1   FULL/DROTHER    00:00:32    172.16.1.4      FastEthernet0/0

fd-wv-ro03#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 172.16.1.1 to network 0.0.0.0

O E1 192.168.29.0/24 [110/10100] via 172.16.1.1, 07:27:01, FastEthernet0/0
O    192.168.122.0/24 [110/110] via 172.16.1.2, 2d19h, FastEthernet0/0
     192.168.133.0/32 is subnetted, 1 subnets
O E1    192.168.133.10 [110/1100] via 172.16.1.1, 00:41:33, FastEthernet0/0
S    192.168.121.0/24 [1/0] via 10.0.0.5, Tunnel121
C    192.168.151.0/24 is directly connected, FastEthernet0/1.151
O    192.168.120.0/24 [110/110] via 172.16.1.2, 2d19h, FastEthernet0/0
C    192.168.150.0/24 is directly connected, FastEthernet0/1.150
O    192.168.110.0/24 [110/200] via 172.16.1.1, 2d19h, FastEthernet0/0
O E1 192.168.9.0/24 [110/10100] via 172.16.1.1, 05:31:48, FastEthernet0/0
     192.168.126.0/25 is subnetted, 1 subnets
O E1    192.168.126.0 [110/1100] via 172.16.1.2, 2d19h, FastEthernet0/0
S    192.168.111.0/24 [1/0] via 10.0.0.9, Tunnel111
O    192.168.125.0/24 [110/110] via 172.16.1.2, 2d19h, FastEthernet0/0
O    192.168.130.0/24 [110/200] via 172.16.1.3, 2d21h, FastEthernet0/0
     172.16.0.0/24 is subnetted, 1 subnets
C       172.16.1.0 is directly connected, FastEthernet0/0
O    192.168.124.0/24 [110/110] via 172.16.1.2, 2d19h, FastEthernet0/0
O    192.168.131.0/24 [110/200] via 172.16.1.3, 2d21h, FastEthernet0/0
O    192.168.140.0/24 [110/200] via 172.16.1.4, 2d20h, FastEthernet0/0
O    192.168.141.0/24 [110/200] via 172.16.1.4, 2d20h, FastEthernet0/0
O E1 192.168.5.0/24 [110/10100] via 172.16.1.1, 06:12:23, FastEthernet0/0
     10.0.0.0/30 is subnetted, 2 subnets
C       10.0.0.8 is directly connected, Tunnel111
C       10.0.0.4 is directly connected, Tunnel121
O    192.168.113.0/24 [110/200] via 172.16.1.1, 2d19h, FastEthernet0/0
O E1 192.168.188.0/24 [110/10100] via 172.16.1.1, 1d00h, FastEthernet0/0
O    192.168.112.0/24 [110/200] via 172.16.1.1, 2d19h, FastEthernet0/0
O E1 192.168.86.0/24 [110/10100] via 172.16.1.1, 06:10:03, FastEthernet0/0
O*E1 0.0.0.0/0 [110/142] via 172.16.1.1, 01:45:39, FastEthernet0/0

Cisco ASA

The Cisco ASA 5505 in my lab runs at version 9.1(4).

I configured the ASA through the ASDM GUI. In the following configuration screenshots, the redistribution of the static routes to the AnyConnect RA VPN are also shown:

And these are some monitoring screenshots:

Juniper SSG

In my lab, it’s an SSG 5 with software version 6.3.0r17.0.

Here with the redistribution of static routes for the Site-to-Site VPNs (complicated: access list, route map, OSPF redistributable rules) and the default route:

And here are a few listings from the CLI. (For some reasons, the host route to the AnyConnect VPN Client on the Cisco ASA, 192.168.133.10/32,  is missing in the routing table. I do not know why. On the Cisco routers as well as on the Palo Alto it is present.)

fd-wv-fw01-> get vrouter trust-vr protocol ospf neighbor
VR: trust-vr RouterId: 172.16.1.1
----------------------------------
                Neighbor(s) on interface ethernet0/5.1 (Area 0.0.0.0)

                Neighbor(s) on interface ethernet0/5.10 (Area 0.0.0.0)

                Neighbor(s) on interface ethernet0/5.2 (Area 0.0.0.0)

                Neighbor(s) on interface ethernet0/5.3 (Area 0.0.0.0)

                Neighbor(s) on interface ethernet0/6 (Area 0.0.0.0)
IpAddr/IfIndex  RouterId        Pri State    Opt  Up           StateChg
------------------------------------------------------------------------------
172.16.1.5      172.16.1.5        1 Full     E    2d;20:30:14  (+7 -0)
172.16.1.3      172.16.1.3        1 2Way     E    2d;20:30:18  (+3 -0)
172.16.1.2      172.16.1.2       50 Full     E    2d;20:30:19  (+7 -0)
172.16.1.4      172.16.1.4        1 2Way     E    2d;20:30:23  (+3 -0)

fd-wv-fw01-> get route v4 protocol ospf


IPv4 Dest-Routes for  (0 entries)
--------------------------------------------------------------------------------------
H: Host C: Connected S: Static A: Auto-Exported
I: Imported R: RIP/RIPng P: Permanent D: Auto-Discovered
N: NHRP
iB: IBGP eB: EBGP O: OSPF/OSPFv3 E1: OSPF external type 1
E2: OSPF/OSPFv3 external type 2 trailing B: backup route


IPv4 Dest-Routes for  (40 entries)
--------------------------------------------------------------------------------------
         ID          IP-Prefix      Interface         Gateway   P Pref    Mtr     Vsys
--------------------------------------------------------------------------------------
         98   192.168.151.0/24         eth0/6      172.16.1.5   O   60    200     Root
*        97   192.168.150.0/24         eth0/6      172.16.1.5   O   60    200     Root
         96   192.168.131.0/24         eth0/6      172.16.1.3   O   60    200     Root
*       102   192.168.130.0/24         eth0/6      172.16.1.3   O   60    200     Root
         94   192.168.141.0/24         eth0/6      172.16.1.4   O   60    200     Root
*        93   192.168.140.0/24         eth0/6      172.16.1.4   O   60    200     Root
*        99   192.168.126.0/25         eth0/6      172.16.1.2  E1   60   1100     Root
*        92   192.168.125.0/24         eth0/6      172.16.1.2   O   60    110     Root
*        91   192.168.124.0/24         eth0/6      172.16.1.2   O   60    110     Root
*        90   192.168.122.0/24         eth0/6      172.16.1.2   O   60    110     Root
         89   192.168.121.0/24         eth0/6      172.16.1.2   O   60    110     Root
*        88   192.168.120.0/24         eth0/6      172.16.1.2   O   60    110     Root

Total number of ospf routes: 12

fd-wv-fw01-> get vrouter trust-vr protocol ospf config
VR: trust-vr RouterId: 172.16.1.1
----------------------------------
set protocol ospf
set enable
set advertise-def-route metric 42 metric-type 1
exit
set protocol ospf
set redistribute route-map "map_redist-vpns" protocol static
exit
set interface ethernet0/5.1 protocol ospf area 0.0.0.0
set interface ethernet0/5.1 protocol ospf passive
set interface ethernet0/5.1 protocol ospf enable
set interface ethernet0/5.1 protocol ospf cost 100
set interface ethernet0/5.10 protocol ospf area 0.0.0.0
set interface ethernet0/5.10 protocol ospf passive
set interface ethernet0/5.10 protocol ospf enable
set interface ethernet0/5.10 protocol ospf cost 100
set interface ethernet0/5.2 protocol ospf area 0.0.0.0
set interface ethernet0/5.2 protocol ospf passive
set interface ethernet0/5.2 protocol ospf enable
set interface ethernet0/5.2 protocol ospf cost 100
set interface ethernet0/5.3 protocol ospf area 0.0.0.0
set interface ethernet0/5.3 protocol ospf passive
set interface ethernet0/5.3 protocol ospf enable
set interface ethernet0/5.3 protocol ospf cost 100
set interface ethernet0/6 protocol ospf area 0.0.0.0
set interface ethernet0/6 protocol ospf enable
set interface ethernet0/6 protocol ospf priority 100
set interface ethernet0/6 protocol ospf cost 100

Palo Alto

Finally, the Palo Alto PA-200 in my lab runs at PAN-OS version 6.0.3.

Before we start, remember to add a security policy rule to allow OSPF on the specific zone. I have forgotten it and was searching a while in all OSPF configurations before I saw the denied packets in the traffic log.

Here are the configuration steps for the OSPF routing. I also configured a redistribution profile which is referenced in the export rules of the OSPF process:

The “More Runtime Stats” look like that:

And for the friends of the CLI, take one of these commands:

show routing protocol ospf neighbor
show routing protocol ospf interface
show routing protocol ospf summary
show routing route type ospf

Links

• Cisco: Configuring OSPF (IOS)
• Cisco: Configuring OSPF (ASA)
• Juniper: Configuring OSPF on firewall
• Corelan Team: Using OSPF on Juniper Netscreen Firewalls
• Palo Alto: Understanding Route Redistribution and Filtering

While preparing for some Palo Alto Networks certifications I read something about the antivirus capabilities of blocking viruses via email by sending an SMTP response code of 541 to the sender (link). This was new for me since I thought the Palo Alto would only block IP connections (TCP RST) but not send layer 7 messages (SMTP codes). But actually, it does so by spoofing the IP address of the destination SMTP host. Cool stuff. Of course, I needed to test this. Here we go.

Basically, I sent an email with a virus through the firewall in order to see if it blocks it.

What is Code 541?

At a first glance I was wondering about the “541” response code. I have not heard about it until now. I have not found any good resources about that on Google. There is nothing in RFC 3463 (Enhanced Mail System Status Codes). Hm. At least, here is a german site that lists this code with a description as “Recipient Address Rejected – Blacklist, Anti-Spam, Mailfilter/Firewall Block”.

Antivirus Profile with Block

The default antivirus profile has an action for smtp of “alert”. In order to block viruses, “block” must be specified:

Of course, this antivirus profile must be used in the appropriate security policy rules for the SMTP mail transfer.

EICAR Testfile sent through PA

I sent a virus (EICAR testfile) from my mail server (Postfix) to an external email address. This SMTP connection goes through the Palo Alto to my MTA (Cisco ESA). Due to the antivirus profile on the PA, this email is blocked. Following are a few listing of this behaviour:

This is the mail.log on the Postfix installation. Note line 6, the longest line, which says “status=bounced [...] 541 5.4.1 Content blocked by Palo Alto Networks Firewall”:

Aug 27 11:26:46 jw-vm10 postfix/smtpd[23809]: connect from unknown[192.168.125.10]
Aug 27 11:26:47 jw-vm10 postfix/smtpd[23809]: 022F115F9BA: client=unknown[192.168.125.10]
Aug 27 11:26:47 jw-vm10 postfix/cleanup[23812]: 022F115F9BA: message-id=<53FDA44D.8010902@webertest.net>
Aug 27 11:26:47 jw-vm10 postfix/qmgr[25919]: 022F115F9BA: from=<johannes@webertest.net>, size=1146, nrcpt=1 (queue active)
Aug 27 11:26:47 jw-vm10 postfix/smtpd[23809]: disconnect from unknown[192.168.125.10]
Aug 27 11:26:48 jw-vm10 postfix/smtp[23813]: 022F115F9BA: to=<johannes@webernetz.net>, relay=2003:51:6012:124::15[2003:51:6012:124::15]:25, delay=0.83, delays=0.2/0.05/0.57/0, dsn=5.4.1, status=bounced (host 2003:51:6012:124::15[2003:51:6012:124::15] said: 541 5.4.1 Content blocked by Palo Alto Networks Firewall (in reply to end of DATA command))
Aug 27 11:26:48 jw-vm10 postfix/cleanup[23812]: 1766B16092F: message-id=<20140827092648.1766B16092F@webertest.net>
Aug 27 11:26:48 jw-vm10 postfix/qmgr[25919]: 1766B16092F: from=<>, size=3117, nrcpt=1 (queue active)
Aug 27 11:26:48 jw-vm10 postfix/bounce[23814]: 022F115F9BA: sender non-delivery notification: 1766B16092F
Aug 27 11:26:48 jw-vm10 postfix/qmgr[25919]: 022F115F9BA: removed
Aug 27 11:26:48 jw-vm10 postfix/local[23816]: 1766B16092F: to=<johannes@webertest.net>, relay=local, delay=0.37, delays=0.16/0.01/0/0.21, dsn=2.0.0, status=sent (delivered to maildir)
Aug 27 11:26:48 jw-vm10 postfix/qmgr[25919]: 1766B16092F: removed

The PA threat log reveals the deny for this connection:

A packet capture on the Postfix servers also shows the SMTP 541 message as well as the TCP RST packet sent from the Palo Alto with an IP address of the real MTA:

Note that the real MTA (in my case a Cisco ESA) sees the incoming connection until it is lost due to the disruption from the Palo Alto:

The sender of the email will get a “Mail Delivery System” email with a subject of “Undelivered Mail Returned to Sender” (or the like). In my test case, this looks like that:

Links

• Palo Alto: CNSE Study Guide
• Palo Alto: Threat Prevention Deployment Tech Note

There are a few application groups that I am almost always using at the customer’s site. These are groups for Microsoft Active Directory, file transfer, and print. Furthermore, I am using a group for all of the Palo Alto Networks management applications itself, a general management group, and two different groups for VPNs (GlobalProtect and site-to-site). Finally, I tested a group for the AVAYA VoIP systems.

Following are the set commands for these groups so that anyone can easily configure them through the CLI.

Some Notes

• These groups are used with PAN-OS 6.0.5 and Apps & Threats version 464-2415. Of course, there should be no problem to use them in later versions.
• I know that they are big in size. They could be smaller for certain applications, such as print or file transfer, if only one kind ot these apps is needed. However, since I use them merely for traffic through internal networks, I am not that much afraid about a rule that allows one or two applications too much.
• I am using the PA-Management group for traffic generated by the management interface of the PA with a direction to internal networks and the Internet.
• The General-Management can be used for many admin accesses.
• The VPN-S2S group is used for site-to-site VPNs. Some of these IPsec tunnels to Cisco devices are recognized as “ciscovpn”. And since “ciscovpn” requires “ssl” and “dtls” to commit without any application-dependency-warnings, they are included in that group, too, though not needed.

Application Groups

To load these application groups into a Palo Alto firewall, enter the

configure

set application-group g_ActiveDirectory [ active-directory dns kerberos ldap ms-ds-smb ms-netlogon ms-wmi msrpc netbios-dg netbios-ns netbios-ss ntp ]
set application-group g_FileTransfer [ ms-ds-smb msrpc netbios-dg netbios-ss nfs portmapper rpc ]
set application-group g_Print [ snmp zpl hp-jetdirect lpd ms-ds-smb msrpc netbios-dg netbios-ss nfs portmapper rpc ]
set application-group g_AVAYA [ avaya-phone-ping dns h.225 h.323 icmp ldap ping rsvp rtcp rtp snmp snmp-trap ssl t.120 traceroute wccp ]
set application-group g_PA-Management [ paloalto-updates paloalto-userid-agent paloalto-wildfire-cloud pan-db-cloud panorama ssl dns ntp brightcloud smtp ]
set application-group g_General-Management [ web-browsing ssl ssh citrix vmware ms-rdp ftp tftp snmp ]
set application-group g_VPN-GlobalProtect [ ike ipsec ssl panos-global-protect panos-web-interface ]
set application-group g_VPN-S2S [ ike ipsec ciscovpn ssl dtls ]

If a mutli vsys environment is used, the set commands must include the “shared” keyword at the beginning:

set shared application-group g_ActiveDirectory [ active-directory dns kerberos ldap ms-ds-smb ms-netlogon ms-wmi msrpc netbios-dg netbios-ns netbios-ss ntp ]
set shared application-group g_FileTransfer [ ms-ds-smb msrpc netbios-dg netbios-ss nfs portmapper rpc ]
set shared application-group g_Print [ snmp zpl hp-jetdirect lpd ms-ds-smb msrpc netbios-dg netbios-ss nfs portmapper rpc ]
set shared application-group g_AVAYA [ avaya-phone-ping dns h.225 h.323 icmp ldap ping rsvp rtcp rtp snmp snmp-trap ssl t.120 traceroute wccp ]
set shared application-group g_PA-Management [ paloalto-updates paloalto-userid-agent paloalto-wildfire-cloud pan-db-cloud panorama ssl dns ntp brightcloud smtp ]
set shared application-group g_General-Management [ web-browsing ssl ssh citrix vmware ms-rdp ftp tftp snmp ]
set shared application-group g_VPN-GlobalProtect [ ike ipsec ssl panos-global-protect panos-web-interface ]
set shared application-group g_VPN-S2S [ ike ipsec ciscovpn ssl dtls ]

That’s it. Did I miss something? If so, send me a comment. Thanks.

You often have comparisons of both firewalls concerning security components. Of course, a firewall must block attacks, scan for viruses, build VPNs, etc. However, in this post I am discussing the advantages and disadvantages from both vendors concerning the management options: How to add and rename objects. How to update a device. How to find log entries. Etc.

Cisco ASA

• Fast Management Suite: The ASDM GUI is really fast. You do not have to wait for the next window if you click on a certain button. It simply appears directly. On the Palo, each entry to add, e.g., an application inside a security rule, takes a few seconds.
• Better “Preview CLI Commands”: I am always checking the CLI commands before I send them to the firewall. On the Cisco ASA, they are quite easy to understand. I know, Palo Alto also offers the “Preview Changes”, but it takes a bit more time to recognize all XML paths.
• Better CLI Commands at all: For Cisco admins it is very easy to parse a “show run” and to paste some commands into another device. This is not that easy on a Palo Alto firewall. First, you must change the config-output format, and second, you cannot simply paste many lines into another device, since the ordering of these lines is NOT correct by default. That is, it simply doesn’t work.
• ACL Hit Count: I like the hit counts per access list entry in the GUI. It quickly reveals which entries are used very often and which ones are never used. On the Palo, you can only highlight the never used ones.
• Many SNMP OIDs: There are many options to monitor the ASA via SNMP. On the Palo Alto, e.g., you can not monitor sub-interfaces. This is really bad. Only the bare metal ethernet ports reveal counters.

Palo Alto PA

• Out-of-Band Management Interface: Even the smallest PA-200 device has its own management interface with its own routing table (default route). This makes it easier to permit/deny admin accesses to this host. E.g., there is no confusion between an access to the SSL VPN and an access to the management GUI since they reside on different interfaces and IP addresses.
• Browser-based GUI: No Java, no client. Just a simple browser. It is also manageable through SSL VPN portals.
• In-Band Interface Management Profiles: On the ASA, every access through different interfaces and different protocols needs its own line to be configured (Management Access -> ASDM/HTTPS/Telnet/SSH). Management access is denied per default, while ping is allowed by default. Both must be set in different menus. Not on the Palo: Interface Mgmt with a few clicks and optional IP addresses, configurable on several interfaces.
• –> Single Security Policy: All interfaces AND site-to-site VPNs are in zones. All security policies between these zones are in one security policy. On the ASA, you don’t have the ACLs for the VPNs in the ACL view of the interfaces since you must specify extra ACLs to the group policy of the VPN.
• Zone Based Security Policies: A policy from zone A to zone B only takes effect for this pair of zones. The “incoming interface” policies on the ASA always have a destination of “any” zone. Though the destination addresses can be limited, it is more complicate to configure the policies if there are several interfaces in use (and not only inside and outside).
• Network Objects in Slash-Notation: Add a host or a network object by typing “1.2.3.0/24″. On the ASA, you have three fields for the same object: host or network, IP address of the network, and netmask (in 255.x.x.x notation!) for the network.
• –> Managing all Un-Commited Changes: One of the best features! Configuration changes can be done in any menu of the Palo Alto, showing the candidate config in all other menus right now, even without a commit. If you rename an object here, it is visible with this new name there. (Try to change the IP-address and the default gateway on a remote Cisco ASA firewall by one step. You won’t succeed until you are using the CLI.)
• Simple Renaming of almost Everything: (Except subinterfaces) Address objects, address groups, zones, security profiles, IPsec tunnels – everything can be renamed. Try to rename an IPsec connection profile on the ASA. Or an interface name. It won’t work or you will get tons of CLI changes.
• History of Configuration Changes: Ever tried to revert to the config from last day? No problem: Load configuration version.
• Configuration Log: Ever wondered who changed something? Here it is: Monitor -> Logs -> Configuration. An exact list of all configuration changes with the name of the administrator.
• Config Audit: Comparison of two configurations, such as of the running-config and any other historical config on the device. Great feature to find certain configuration changes.
• –> Traffic Log Filtering: This is one of the MAJOR advantages of a Palo Alto GUI. It is really simple to click some objects to filter the traffic log. Or to build more precise filters. “eq” and “neq” are your friends. Forget the Real-Time Log Viewer from Cisco.
• Adjust Columns: Or even the possibility to adjust the columns. On the ASDM GUI from Cisco, some pages are per default to small to show the relevant values, e.g., the Monitoring -> Routing -> Routes pane.
• Application Command Center: A simple but useful monitoring tool within the GUI. You are searching for the IP that generates high traffic load during the last hour? Here you will find it. What source country is responsible for the attacks during the last week? Here you go.
• –> Route-Based VPN: A site-to-site VPN connection is built by two gateways, independent of the traffic being routed through the tunnel. Numbered tunnel-interfaces can be used to ping the tunnel endpoint of the other side. The decision where to route the traffic is based on the routing table and not on a policy. The Cisco firewall uses policy-based VPNs in which the Proxy-IDs per connection define the tunneled networks. A bit unhandy.
• –> IKE Policy per VPN: Every gateway has its own IKE profile configured. Different IKE settings can be used for different VPNs. The Cisco has global IKE parameters.
• Reasonable Default Crypto Settings: The default groups for the IPsec phase 1 and phase 2 crypto profiles have almost secure settings. Very good compared to the Cisco ASA, which really installs a view default profiles, e.g., an IKE policy with an encryption algorithm of “DES”. Yes, not 3DES, but only simple DES! Oh oh.
• Retrieve License Keys from Server: Really cool feature. And very easy to use for the customer. Once the authorization code is added in the Palo Alto support portal, the firewall can retrieve its license via https. No need for any further activation keys.
• Built-In Software Archive: Firmware versions can be downloaded directly through the GUI. No need for further logins, downloads from the vendor page and uploads to the unit. Just “Download” and “Install”.
• Enough Disk Space for several Softwares: On my (small) Cisco ASA 5505, the built-in flash disk has only 128 MB. That is, I cannot even do a simple software upgrade because the free disk space does not fit for two ASA images. (I have an ASA and ASDM image as well as three AnyConnect images on the fash memory.) What a mess!
• Sync Software to HA Member: Every software that is downloaded on the primary firewall can automatically be synced to the secondary device. This is not true on the Cisco ASA, which is really annoying when it comes to AnyConnect remote access VPN client images. If these are not uploaded manually on the second device, the other HA unit will not terminate VPN tunnels in case of a HA active-unit swap. Oh oh!
• HA Status in GUI: With the High Availability widget, the status of the HA is visualized with green/orange/red bubbles. It shows which unit is the active/standby one. Since the PA has a real OoB management, the admin can access both devices simultaneously and can see which hardware is the active and the passive one. The Cisco ASA swaps its IP addresses and has no OoB management, so it is harder to see which hardware is the primary and the secondary one, since its IP addresses swap, too.
• NTP Servers with Names: I know that NTP servers should be set via IP addresses to not rely on another service (DNS), but it is much more easier to use names such as

  de.pool.ntp.org

  or the like. This can be done on the Palo Alto, but not on the Cisco firewall.
• No “bring to top” GUI: During the start of Cisco’s ASDM, it always brings its GUI to the top of all windows. In my opinion, this is annoying. During the 30-60 seconds until the whole device config is loaded into the GUI, I am working on other things. But these are generally disrupted from the highlighting of the ASDM GUI. This does not happen with the Palo Alto GUI which is in one tab of my browser.

(The major advantages are marked with an –> arrow.)

Summary

In summary, I really love the management GUI from the Palo Alto. Not hard due to the list of more than 20 advantages over the Cisco ASA platform. Though it is slower than the ASDM GUI from Cisco, it offers much more useful capabilities for the daily work. Great!

© Image Sketch by Mareike Weber – Kunst und so

Here is my MRTG/Routers2 configuration for a Palo Alto Networks PA-200 firewall. It uses all available OIDs from the PAN-MIB. With a few search-and-replace runs, this template can be used in many other scenarios.

SNMP Tests

In my testbed, I am using a PA-200 with PAN-OS 6.1.1. That is, I used the Enterprise SNMP MIB 6.1 from Palo Alto. This is relevant to know since Palo Alto changed a few OIDs from PAN-OS version 5.0.x to 6.0.x.

Note that the following template must be adjusted if it is used with other Palo Alto firewalls, e.g., when more than CPU or multiple VSYS are used.

Unfortunately, the PA firewall is very limited when it comes to monitoring it via SNMP. Is has only a single MIB with a few values. Furthermore, the interfaces cannot be monitored as known from other firewall vendors. For example, there are no counters for subinterfaces or for VLAN interfaces. This is really bad! Furthermore, no site-to-site VPN statistics can be read out, and so on. However, at least the speed of the fan can be requested.

The following values are accessible via SNMP:

• CPU of the data- and management-plane
• Disk space of all partitions
• Fan speed
• GlobalProtect tunnels
• Sessions: ICMP, SSL, TCP, UDP
• Temperature
• Memory: real and swap
• Interfaces: all data ports + management port

My MRTG/Routers2 Configuration

At first, I ran the cfgmaker to get the interfaces. I am also using two global options: one for the icon and one for the “mirror” graph style:

cfgmaker --snmp-options=:::::2 --show-op-down --zero-speed=1000000000 --global "routers.cgi*Icon: firewall3-sm.gif" --global "routers.cgi*GraphStyle[_]: mirror" --output=NAMEOFTHEFIREWALL.cfg COMMUNITY@192.168.120.2

Then, as always, I deleted the Global Config Options except the two ones that were added through the global options with cfgmaker. Furthermore, the “noHC[…]: yes” lines (if present) can be deleted.

For all specific Palo Alto OIDs, use the following template and copy the contents into the just generated cfg file. Of course, the targets for the interfaces should not be copied. Read the first lines of that file to know which values must be adjusted.

MRTG-Routers2 Template Palo Alto 17.50 KB

Download cfg

Sample Graphs

This leads to the following graphs (here in the monthly view):

Links

• Palo Alto: Enterprise SNMP MIB 6.1
• Palo Alto: Enterprise SNMP MIB 6.0
• Palo Alto: Enterprise SNMP MIB 5.0
• Palo Alto: Useful SNMP OIDs for Monitoring Palo Alto Networks Devices
• Palo Alto: What MIBs supported on the PAN?

… the whole Internet breaks down. So happened on a Palo Alto with a DNS proxy and a (slightly misconfigured) anti-spyware profile.

All network clients had a single DNS server configured, namely the DNS proxy of the Palo Alto. And as a single network client requested an URL that was classified as “spyware”, the Palo correctly (!) blocked the DNS session from its DNS proxy to the Internet. Unluckily, this session stayed active for a long time (with drop-all-packets) since many DNS requests were traversing through it. But since it stayed blocked, the Internet was “unavailable” for all end users

Here is a screenshot of that session. Note the duration of it (more than 1 h) and the threat that was recognized as spyware due to the malicious DNS request. The bytes received (57,341) were much smaller than the bytes sent (23,008,857) which shows that many DNS requests were initiated but never answered. The packets sent counter had a value of more than 200.000 while the packets received had only 286.

The quick solution is to manually clear this session. Either via the GUI (Monitor -> Session Browser) or via the CLI (

clear session id <id>

A long-term solution is to have a unique Anti-Spyware Profile with a DNS signatures action of “alert” for the outgoing DNS requests (= DNS proxy to Internet DNS servers), while the profile for the incoming DNS requests (= user to DNS proxy) has a profile with “block”. That is, malicious requests from a user only block the DNS session from that particular user while the main DNS proxy session stays active and functional.

This is a small tutorial for configuring a site-to-site IPsec VPN between a Palo Alto and a FortiGate firewall. I am publishing step-by-step screenshots for both firewalls as well as a few troubleshooting CLI commands.

Lab

This is my basic laboratory for this VPN connection. I am using a Palo Alto PA-200 with PAN-OS 6.1.1 while the FortiWiFi 90D has v5.2.2 installed.

Palo Alto

The Palo Alto is configured in the following way. Please refer to the descriptions under the images for detailed information.

(And do not forget the “untrust-untrust” policy that allows ipsec!)

FortiGate

And this is the way for the FortiGate firewall:

Monitoring

Following are a few screenshots and listings from both firewalls concerning the VPN:

Palo Alto CLI:

weberjoh@fd-wv-fw02> show vpn ike-sa gateway fd-wv-fw04

phase-1 SAs
GwID/client IP  Peer-Address           Gateway Name           Role Mode Algorithm          Established     Expiration      V  ST Xt Phase2
--------------- ------------           ------------           ---- ---- ---------          -----------     ----------      -  -- -- ------
             10 172.16.1.6             fd-wv-fw04             Resp Main PSK/DH14/A256/SHA256 Jan.20 11:12:57 Jan.20 19:12:57 v1 12  2      1

Show IKEv1 IKE SA: Total 1 gateways found. 1 ike sa found.

phase-2 SAs
GwID/client IP  Peer-Address           Gateway Name           Role Algorithm               SPI(in)  SPI(out) MsgID    ST Xt
--------------- ------------           ------------           ---- ---------               -------  -------- -----    -- --
             10 172.16.1.6             fd-wv-fw04             Resp DH14/tunl/ESP/A256/SHA256 A3D05151 C97B0AB3 C5572823  9  1

Show IKEv1 phase2 SA: Total 1 gateways found. 1 ike sa found.

FortiGate CLI:

fd-wv-fw04 # get vpn ike gateway fd-wv-fw02

vd: root/0
name: fd-wv-fw02
version: 1
interface: wan1 6
addr: 172.16.1.6:500 -> 172.16.1.2:500
created: 572s ago
IKE SA  created: 1/1  established: 1/1  time: 70/70/70 ms
IPsec SA  created: 1/1  established: 1/1  time: 90/90/90 ms

  id/spi: 20057 2b5ce64a51119571/defa8a4a3a5f0573
  direction: initiator
  status: established 572-572s ago = 70ms
  proposal: aes-256-sha256
  key: ed29b2dc34c59b46-c587e9daee5d91fb-d83448f2f91bcbae-60505b8efc09fb72
  lifetime/rekey: 28800/27927
  DPD sent/recv: 0000006e/00000000

fd-wv-fw04 #
fd-wv-fw04 #
fd-wv-fw04 # get vpn ipsec tunnel name fd-wv-fw02

gateway
  name: 'fd-wv-fw02'
  type: route-based
  local-gateway: 172.16.1.6:0 (static)
  remote-gateway: 172.16.1.2:0 (static)
  mode: ike-v1
  interface: 'wan1' (6)
  rx  packets: 641  bytes: 65776  errors: 0
  tx  packets: 642  bytes: 168  errors: 1
  dpd: enabled/negotiated  idle: 5000ms  retry: 3  count: 0
  selectors
    name: 'fd-wv-fw02'
    auto-negotiate: disable
    mode: tunnel
    src: 0:0.0.0.0/0.0.0.0:0
    dst: 0:0.0.0.0/0.0.0.0:0
    SA
      lifetime/rekey: 3600/2907
      mtu: 1438
      tx-esp-seq: 280
      replay: enabled
      inbound
        spi: c97b0ab3
        enc:     aes  51128eb018d1ba7bc1e701f2c98689895df63dd1ca0c0252a07b178c5b867652
        auth: sha256  66b3dee1523d2aefd008e3d350a140133b76ebcb768974d6142c4d2f118c0862
      outbound
        spi: a3d05151
        enc:     aes  f168c4dc08b795dc978a4def5979acdc4aa7fbf0bedc1a1c4271bd2cbfe76f40
        auth: sha256  353c678f8ff1e782b3c59eed1628e80a574f2162f4ac5d38cbffe58e538dd064
      NPU acceleration: encryption(outbound) decryption(inbound)

A few month ago I found a small bug in PANOS, the operating system from Palo Alto Networks. It is related to an IPv6 enabled management interface. The MGT address was not reachable when the firewall operates in layer 2 mode, that is, had layer 2 interfaces along with VLANs. Luckily, this bug is fixed with the new software version 6.1.2 which was released this week (bug ID 67719).

Following are a few listings that show the incomplete handling of the IPv6 neighbor cache of the MGT interface in the old version (pre 6.1.2).

I was using the layer 2 mode for some switch tests about STP. During these tests I noticed that I was not able to connect to the MGT interface via IPv6 anymore.

The Palo Alto in my lab has a VLAN interface (vlan.120) and the corresponding VLAN on a layer 2 subinterface. The management port is plugged into a switch in the same VLAN. The IPv6 address on the MGT interface is

2003:51:6012:120::2/64

Bug

For example, when trying to ping or to ssh to the MGT interface from another machine …

weberjoh@jw-nb08:~$ ping6 2003:51:6012:120::2
PING 2003:51:6012:120::2(2003:51:6012:120::2) 56 data bytes
^C
--- 2003:51:6012:120::2 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5039ms

weberjoh@jw-nb08:~$
weberjoh@jw-nb08:~$
weberjoh@jw-nb08:~$ ssh -v pa-mgmt.webernetz.net
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to pa-mgmt.webernetz.net [2003:51:6012:120::2] port 22.
^C

… the neighbor cache did not show the MGT IPv6 address:

weberjoh@fd-wv-fw02> show neighbor vlan.120

maximum of entries supported :      500
default base reachable time:        30 seconds
total neighbor entries in table :   27
total neighbor entries shown :      7

interface         ip address                              hw address        status
--------------------------------------------------------------------------------
vlan.120          2003:51:6012:120::10                    00:1d:92:53:58:12 STALE
vlan.120          2003:51:6012:120::13                    00:0c:29:be:67:4d STALE
vlan.120          fe80::20c:29ff:febe:674d                00:0c:29:be:67:4d STALE
vlan.120          fe80::20c:29ff:fefb:69c4                00:0c:29:fb:69:c4 STALE
vlan.120          fe80::219:e2ff:fea1:f986                00:19:e2:a1:f9:86 STALE
vlan.120          fe80::21d:92ff:fe53:5812                00:1d:92:53:58:12 STALE
vlan.120          fe80::b60c:25ff:fe05:8e00               b4:0c:25:05:8e:00 STALE

However, I was able to ping from that MGT interface IPv6 address. Interestingly, the neighbor cache revealed the ::2 address, but only with the status “PROBE” and only for a very few seconds:

weberjoh@fd-wv-fw02> ping inet6 yes source 2003:51:6012:120::2 host heise.de
PING heise.de(redirector.heise.de) from 2003:51:6012:120::2 : 56 data bytes
64 bytes from redirector.heise.de: icmp_seq=0 ttl=54 time=72.8 ms
64 bytes from redirector.heise.de: icmp_seq=1 ttl=54 time=24.8 ms
64 bytes from redirector.heise.de: icmp_seq=2 ttl=54 time=22.0 ms
64 bytes from redirector.heise.de: icmp_seq=3 ttl=54 time=26.4 ms
^C
--- heise.de ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3029ms
rtt min/avg/max/mdev = 22.081/36.543/72.831/21.008 ms, pipe 2

weberjoh@fd-wv-fw02> show neighbor vlan.120

maximum of entries supported :      500
default base reachable time:        30 seconds
total neighbor entries in table :   27
total neighbor entries shown :      7

interface         ip address                              hw address        status
--------------------------------------------------------------------------------
vlan.120          2003:51:6012:120::2                     b4:0c:25:05:8e:00 PROBE
vlan.120          2003:51:6012:120::13                    00:0c:29:be:67:4d STALE
vlan.120          fe80::20c:29ff:febe:674d                00:0c:29:be:67:4d STALE
vlan.120          fe80::20c:29ff:fefb:69c4                00:0c:29:fb:69:c4 STALE
vlan.120          fe80::219:e2ff:fea1:f986                00:19:e2:a1:f9:86 STALE
vlan.120          fe80::21d:92ff:fe53:5812                00:1d:92:53:58:12 STALE
vlan.120          fe80::b60c:25ff:fe05:8e00               b4:0c:25:05:8e:00 STALE

weberjoh@fd-wv-fw02> show neighbor vlan.120

maximum of entries supported :      500
default base reachable time:        30 seconds
total neighbor entries in table :   26
total neighbor entries shown :      6

interface         ip address                              hw address        status
--------------------------------------------------------------------------------
vlan.120          2003:51:6012:120::13                    00:0c:29:be:67:4d STALE
vlan.120          fe80::20c:29ff:febe:674d                00:0c:29:be:67:4d STALE
vlan.120          fe80::20c:29ff:fefb:69c4                00:0c:29:fb:69:c4 STALE
vlan.120          fe80::219:e2ff:fea1:f986                00:19:e2:a1:f9:86 STALE
vlan.120          fe80::21d:92ff:fe53:5812                00:1d:92:53:58:12 STALE
vlan.120          fe80::b60c:25ff:fe05:8e00               b4:0c:25:05:8e:00 STALE

The traffic log on the Palo Alto shows that incoming connections did not succeed, while outgoing connections did:

Fixed in 6.1.2

with bug ID 67719: “The management interface was not receiving IPv6 connections for traffic from the dataplane when the firewall was in Layer 2 mode. An update was made to the MAC address learning process so that the Management interface receives IPv6 traffic from the dataplane when the firewall is in Layer 2 mode.”

Now I can ping to the IPv6 MGT address:

weberjoh@jw-nb08:~$ ping6 2003:51:6012:120::2
PING 2003:51:6012:120::2(2003:51:6012:120::2) 56 data bytes
64 bytes from 2003:51:6012:120::2: icmp_seq=1 ttl=62 time=1.54 ms
64 bytes from 2003:51:6012:120::2: icmp_seq=2 ttl=62 time=1.05 ms
64 bytes from 2003:51:6012:120::2: icmp_seq=3 ttl=62 time=1.17 ms
64 bytes from 2003:51:6012:120::2: icmp_seq=4 ttl=62 time=1.16 ms
^C
--- 2003:51:6012:120::2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 1.056/1.235/1.547/0.189 ms

And the neighbor cache correctly shows the REACHABLE/STALE neighbor:

weberjoh@fd-wv-fw02> show neighbor vlan.120

maximum of entries supported :      500
default base reachable time:        30 seconds
total neighbor entries in table :   10
total neighbor entries shown :      2

interface         ip address                              hw address        status
--------------------------------------------------------------------------------
vlan.120          2003:51:6012:120::2                     b4:0c:25:05:8e:00 STALE
vlan.120          fe80::b60c:25ff:fe05:8e00               b4:0c:25:05:8e:00 STALE

Another fixed issue in the just released PANOS version 6.1.2 from Palo Alto Networks is bug ID 71321: “Removed support for SSL 3.0 from the GlobalProtect gateway, GlobalProtect portal, and Captive Portal due to CVE-2014-3566 (POODLE).” I scanned my lab unit before (6.1.1) and after the OS upgrade (6.1.2) and here are the results.

Once more I am using the Qualys SSL Server Test to test the TLS status of my services, in this case the Palo Alto GlobalProtect login page. Here are the two results before and after the update to version 6.1.2. Since the previous version was vulnerable to the “Padding Oracle On Downgraded Legacy Encryption” attack, the overall rating was degraded to F.

Though it is nice that the TLS connections to the Palo Alto are not vulnerable to these types of attacks anymore, I would prefer to choose the protocols and ciphers that are used on the server, and not only to rely on the default Palo Alto settings. E.g., there is not single cipher available that supports Perfect Forward Secrecy. Oh oh.

When working with Cisco devices anyone knows that the output of a “show running-config” on one device can be used to completely configure a new device. On a Palo Alto Networks firewall, this is not that obvious. There are several commands that must be used to achieve the same.

However, I tested this procedure a few times and it did NOT work. So, the short version is: If you want to replace a Palo Alto firewall, move your configuration files (xml) through the GUI or tftp/scp. But do not use the mere CLI.

The most common way to save a Palo Alto config is via the GUI at Device -> Setup -> Operations ->  Export xyz. And even on the CLI, the running-config can be transferred via scp or tftp, such as

scp export configuration from running-config.xml to username@host:path

scp import configuration from username@host:path

Save

However, to save the complete configuration in the “set” format, the following CLI commands must be used. The first one is used to output the configuration in single “set” lines (instead of XML blocks), and the second one switches the output to not stop after a few lines on the terminal. To capture long lines without a “carriage return”, the terminal width should be adjusted to the maximum of 500. Then, the “configure” command enters the configuration mode, while the “show” command displays the whole running configuration.

> set cli config-output-format set
> set cli pager off
> set cli terminal width 500
> configure
# show

And Load

To load the config into a new device, a few commands must be used before. At first, the terminal width should be adjusted again. Furthermore, the scripting-mode must be enabled in order to send a bulk of CLI commands without an error. The reason for that is, that several objects are referenced in the configuration before they are added to the device. E.g., the set commands for the “security rules” are before the set commands for the “application groups”. That is, an application group is used by a security rule before it is added to the config.

Finally, the whole bunch of set commands from above can be pasted into the CLI session.

> set cli terminal width 500
> set cli scripting-mode on
> configure
# set ...

Errors, Errors, Errors

I wanted to load a complete configuration from a firewall to another. (Both firewalls were of the same type, OS version (6.0.x) and license.) I used the console port on the device. But even with the aforementioned commands that should make this procedure possible, I got only errors, such as: “Invalid syntax.” or “Unknown command: …”.

Furthermore, the terminal session looked like a complete chaos:

admin@PA-5050# 
Server error :  -> from 'trustL3' is not an allowed keyword
 -> from 'trustL3' is not a valid reference
ir-update firefox-update google-update java-upd[edit]                                                                                                                                                                           
admin@PA-5050# 
Server error :  -> to 'untrustL3' is not an allowed keyword
 -> to 'untrustL3' is not a valid reference
ate ms-update ]
set application-group test-shar[edit]                                                                                                                                                                           
admin@PA-5050# ed-appgroup [ 2ch-posting 4shared ]
set appli
[edit]

Conclusion

Only use the complete XML-based configuration files and not the set commands!

The Palo Alto firewall has a feature called DNS Proxy. Normally it is used for data plane interfaces so that clients can use the interfaces of the Palo for its recursive DNS server. Furthermore, this DNS Proxy Object can be used for the DNS services of the management plane, specified under Device -> Setup -> Services. However, there was a bug in PAN-OS that did not process the proxy rules and static entries when a DNS proxy object was used in the management plane. This bug was fixed in PAN-OS 6.0.0. I tested it in my lab with PAN-OS 6.1.0 running. Here are the successful results.

The fixed bug in version 6.0.0 was bug ID 41472: “When a DNS Proxy object was configured with static entries, hostnames assigned to the DNS Proxy were resolved as expected to the IP addresses listed on the Static Entries tab (Network > DNS Proxy) . However, when setting the DNS Proxy Object as the DNS Service on the Device > Setup > Services dialog, all DNS queries from the management interface ignored the defined static entries.”

DNS Proxy

I added a DNS proxy called “Google” with their two public DNS servers, as well as a few proxy rules to other DNS servers and static entries. This proxy object was then referenced for the management services as a “DNS Proxy Object”:

Tests

After a commit, I pinged a few hosts from the management plane. The first one should use the default DNS server, while the three following requests should use the proxy rules. The last object was the static entry in the DNS proxy.

weberjoh@fd-wv-fw02> ping host heise.de
PING heise.de (193.99.144.80) 56(84) bytes of data.
64 bytes from redirector.heise.de (193.99.144.80): icmp_seq=1 ttl=245 time=44.0 ms
64 bytes from redirector.heise.de (193.99.144.80): icmp_seq=2 ttl=245 time=12.5 ms
^C
--- heise.de ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1008ms
rtt min/avg/max/mdev = 12.514/28.279/44.044/15.765 ms
weberjoh@fd-wv-fw02> ping host webernetz.net
PING webernetz.net (80.237.133.136) 56(84) bytes of data.
64 bytes from wp367.webpack.hosteurope.de (80.237.133.136): icmp_seq=1 ttl=55 time=86.9 ms
64 bytes from wp367.webpack.hosteurope.de (80.237.133.136): icmp_seq=2 ttl=55 time=66.9 ms
^C
--- webernetz.net ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1004ms
rtt min/avg/max/mdev = 66.965/76.949/86.933/9.984 ms
weberjoh@fd-wv-fw02> ping host ra.webernetz.net
PING ra.webernetz.net (192.168.112.4) 56(84) bytes of data.
64 bytes from 192.168.112.4: icmp_seq=1 ttl=62 time=1.13 ms
64 bytes from 192.168.112.4: icmp_seq=2 ttl=62 time=1.20 ms
^C
--- ra.webernetz.net ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1008ms
rtt min/avg/max/mdev = 1.139/1.170/1.201/0.031 ms
weberjoh@fd-wv-fw02> ping host www.formel1.de
PING www.formel1.de (78.138.119.82) 56(84) bytes of data.
64 bytes from f1-lb01.formel1.de (78.138.119.82): icmp_seq=1 ttl=53 time=11.9 ms
64 bytes from f1-lb01.formel1.de (78.138.119.82): icmp_seq=2 ttl=53 time=52.4 ms
^C
--- www.formel1.de ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1009ms
rtt min/avg/max/mdev = 11.925/32.206/52.487/20.281 ms
weberjoh@fd-wv-fw02> ping host asdf.foobar.com
PING asdf.foobar.com (1.2.3.4) 56(84) bytes of data.
^C
--- asdf.foobar.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

Since the default route of the management interface points through one of the data interfaces of the Palo Alto, I could easily do a packet capture on the Palo itself. It correctly revealed the three different DNS servers for the requests. Of course, the request to the static entry is not shown since it did not trigger a DNS request to a server.

Works as Expected

The tests showed that the function of a DNS proxy for the management services works as expected. Not only the default DNS servers, but also the proxy rules and static entries are used through the management plane.

Note that the following scenarios do NOT work:

• Pre PAN-OS 6.x, the DNS Proxy object used in the management services sent all requests to the primary and secondary DNS servers but NOT to the proxy rules. The static entries were not used, too.
• It is not possible to set a “DNS Server” in the management services to the IP address of a data interface of the Palo Alto itself. This won’t work since the Palo uses some kind of internal path when sending packets to an own data interface. This means, if a DNS Proxy object should be used, it must be selected as a “DNS Proxy Object” and not as an entry of “DNS Servers”.

Since IPv6 gets more and more important, I am using it by default on all my test firewalls, which of course support IPv6. However, when comparing the different functions and administration capabilities, they vary significantly.

Here comes my short evaluation of the IPv6 functions on the following four firewalls: Cisco ASA, Fortinet FortiGate, Juniper SSG, and Palo Alto.

Criteria

I was merely interested in the basic IPv6 usage and not in the typical firewall categories:

• Interface: IPv6 address and link-local address configurable?
• Router Advertisement and DHCPv6: Whether the firewalls support nothing (–), only RA (-), DHCPv6 relay (ο), stateless DHCPv6 (+), or stateful DHCPv6 (++). The existence of stateless DHCPv6 is vital for delivering the DNS server IPv6 addresses to the clients. (The “IPv6 Router Advertisement Options for DNS Configuration”, RFC 6106, is not supported by any of these devices.)
• Security Policy: Whether IPv4 and IPv6 addresses can be used in the same policy and whether address groups can have objects from both protocols.
• Administration: How easy are the IPv6 functions to manage? Only via the CLI (–), fifty-fifty (ο), GUI but complicated (+) , or fully via the GUI (++).

Results

These are the results. They range from — via ο to ++.

Details

Cisco ASA

The Cisco ASA has no DHCPv6 instance running. That is: there is no way to run an IPv6-only network because clients won’t get the DNS server. The security policy is capable of both protocols. Everything is configurable via the GUI, which is not the best at all.

Fortinet FortiGate

The FortiGate is the only firewall with a stateful DHCPv6 server. Great. However, two distinct security policies must be used and nothing of the IPv6 settings are configurable via the GUI. WHAT???

Juniper SSG (ScreenOS)

ScreenOS is dead. However, most of the IPv6 functions are working quite good, except the protocol dependent security policies. Everything is accessible via the GUI, but sometimes on confusing positions.

Palo Alto

Palo Alto did a good job on the IPv6 interfaces and security policies. The GUI is quite intuitive and the policy accepts both protocols at the same time. Unluckily, there is no DHCPv6 server which makes it impossible to operate an IPv6-only client network behind a Palo Alto (without further servers).

Conclusion

It’s interesting to see the differences between those firewalls. While the Fortinet und Juniper firewalls support the whole SLAAC process incl. DNS servers, they have no single security policy for both protocols and are horrable to configure.

The Palo Alto is quite good to configure but lacks the DHCPv6 server. Same for the Cisco.

In summary, all firewalls position in the middle of my scale. From an IPv6-only view, I cannot say which one is the best. It depends….

Beside the HA1 and HA2 interfaces on a Palo Alto Networks firewall, there are the HA1/HA2 Backup and Heartbeat Backup options. I was a bit confused while reading the documentation of the high availability instructions since it did not clearly specify when and where to use the dedicated management port for what kind of “backup”.

Basically, it should read that there are two different ways on how to use the dedicated management for a HA Backup: the heartbeat backup OR the HA1 backup.

(The screenshots are from a PA-5050 with PAN-OS version 6.1.5 running. The official Palo Alto Networks links are here: High Availability Resources.)

I was confused because the Palo Alto documentation says about the heartbeat backup: “Uses the management ports on the HA devices to provide a backup path for heartbeat and hello messages. The management port IP address will be shared with the HA peer through the HA1 control link. No additional configuration is required.”

And for the HA1 backup, it says: “The recommended configuration for the HA control link connection is to use the dedicated HA1 link between the two devices and use the management port as the Control Link (HA Backup) interface. In this case, you do not need to enable the Heartbeat Backup option in the Elections Settings page.”

–> It was not clear for me, where to use the management port and when to enable the heartbeat backup.

Solution

The heartbeat backup option should only be used if the management interface is not used at HA1 or HA1 backup. Otherwise, the heartbeat backup should not be used.

The following two screenshots show the different options. Either the heartbeat backup is activated, or the HA1 backup link is the management interface:

In any case, the HA1 Backup bubble should be green:

Note that the heartbeat backup is ONLY using heartbeat and hello messages, while the HA1/backup link does more: “Control Link: The HA1 link is used to exchange hellos, heartbeats, and HA state information, and management plane sync for routing, and User-ID information. This link is also used to synchronize configuration changes on either the active or passive device with its peer.” –> Choose the option in which the managenent interface is the real HA1 backup and not only the heartbeat backup!

Supplement

Just while searching for all that HA stuff I found a list in the admin guide for version 6.1 that exactly describes when or when not to use the “heartbeat backup” option. On page 143 it lists (among others) this cases:

HA1: Dedicated HA1 port
HA1 Backup: In-band port
Recommendation: Enable Heartbeat Backup

and:

HA1: Dedicated HA1 port
HA1 Backup: Management port
Recommendation: Do not enable Heartbeat Backup

Now it’s clear.