Encryption failure: Received a cleartext packet within an encrypted connection

Recently I’ve stumbled upon a strange looking site-to-site (CheckPoint R70 to Cisco VPN3k) VPN problem:

Connections from some networks were dropped with the following error:

Encryption failure: Received a cleartext packet within an encrypted connection

The first step was to check the encryption domains for the tunnel. In both GUI and /etc/fw/conf/user.def the encryption domain was the whole class B network, assigned to the company.

Next step was tracing.

Non-working example:

[Expert@datgwy04c]# fw monitor -e "accept src=163.157.180.17;"

Internal:i[48]: host1 -> server1 (TCP) len=48 id=53735
TCP: 2316 -> 80 .S.... seq=32dabb6a ack=00000000

Internal:i[48]: host1 -> server1 (TCP) len=48 id=53746
TCP: 2316 -> 80 .S.... seq=32dabb6a ack=00000000

Internal:i[48]: host1 -> server1 (TCP) len=48 id=53763
TCP: 2316 -> 80 .S.... seq=32dabb6a ack=00000000

Extended version:

in chain (16):
         0: -7f800000 (f9a26ee0) (ffffffff) IP Options Strip (in) (ipopt_strip)
         1: - 2000000 (f94b6a90) (00000001) vpn decrypt (vpn)
         2: - 1fffff8 (f94c18c0) (00000001) l2tp inbound (l2tp)
         3: - 1fffff6 (f9a28190) (00000001) Stateless verifications (in) (asm)
         4: - 1fffff2 (f94d5db0) (00000001) vpn tagging inbound (tagging)
         5: - 1fffff0 (f94b78a0) (00000001) vpn decrypt verify (vpn_ver)
         6: - 1000000 (f9a71410) (00000003) SecureXL conn sync (secxl_sync)
         7:         0 (f99e17f0) (00000001) fw VM inbound  (fw)
         8:         1 (f9a3df00) (00000002) wire VM inbound  (wire_vm)
         9:   2000000 (f94b99e0) (00000001) vpn policy inbound (vpn_pol)
         10:  10000000 (f9a76a00) (00000003) SecureXL inbound (secxl)
         11:  7f600000 (f9a1ded0) (00000001) fw SCV inbound (scv)
         12:  7f730000 (f9b42a10) (00000001) passive streaming (in) (pass_str)
         13:  7f750000 (f9c03330) (00000001) TCP streaming (in) (cpas)
         14:  7f800000 (f9a27260) (ffffffff) IP Options Restore (in) (ipopt_res)
         15:  7fb00000 (f9bd2ae0) (00000001) HA Forwarding (ha_for)

Internal:i0 (IP Options Strip (in))[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i1 (vpn decrypt)[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i2 (l2tp inbound)[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i3 (Stateless verifications (in))[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i4 (vpn tagging inbound)[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i5 (vpn decrypt verify)[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i6 (SecureXL conn sync)[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:i7 (fw VM inbound )[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:I8 (wire VM inbound )[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000
Internal:I9 (vpn policy inbound)[48]: host1 -> server1 (TCP) len=48 id=31393
TCP: 3293 -> 21 .S.... seq=b144a49c ack=00000000

The packets did not go past “vpn policy inbound”.

Tcpdump also did not bring much information:

Working example:

# tcpdump -i Internal ip host server1

03:59:04.923978 host2.41810 > server1.ftp: S 3084710943:3084710943(0) win 5840  (DF)
03:59:07.922888 host2.41810 > server1.ftp: S 3084710943:3084710943(0) win 5840  (DF)
03:59:13.922355 host2.41810 > server1.ftp: S 3084710943:3084710943(0) win 5840  (DF)

Non-working example:

03:58:35.727270 163.157.180.17.3280 > server1.ftp: S 3765504144:3765504144(0) win 64512  (DF)
03:58:38.801495 163.157.180.17.3280 > server1.ftp: S 3765504144:3765504144(0) win 64512  (DF)
03:58:44.816236 163.157.180.17.3280 > server1.ftp: S 3765504144:3765504144(0) win 64512  (DF)

What actually helped was “vpn overlap_encdom” command on the management server (the command displays overlapping encryption domains). The networks from which the connections were not working did not show up in the overlapping domains for the firewall cluster. Hence, there were more domain configured I was not aware off.

Further investigation showed that (for historical reasons?) not all networks were included in the following two objects (both with multilevel inclusions, what worsened the troubleshooting):

• VPN domain for the cluster (Network Objects -> Check Point -> FW cluster object -> Details -> Topology -> VPN Domain -> Manually defined)
• Valid-hme1 group. This is a strange one. Because “Valid-hme1” is not used in any policy and there are no such interfaces (hme1) on any of the used firewalls. Looks like old Solaris antispoofing object.