Date: Wed, 25 Mar 2009 09:59:43 +0100 From: Sebastiaan van Erk <sebster@sebster.com> To: freebsd-pf@freebsd.org Subject: state mismatch/connection issues Message-ID: <49C9F27F.3010505@sebster.com>
next in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.
--------------ms030107090002090101050409
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Hi,
I'm running FreeBSD-7.0 RELEASE with the following patch to the kernel
(I know it's integrated in the latest patchlevels which you get when you
do freebsd-update, but since I'm still getting state-mismatches WITH the
patch I'm holding off on the upgrade until I have more information as to
the nature of the problem):
*** net/pf.c 2007/09/07 21:34:10 1.558
--- net/pf.c 2007/09/18 19:45:59 1.559
*************** pf_test_state_tcp(struct pf_state **state, int directi
*** 3730,3735 ****
--- 3730,3751 ----
REASON_SET(reason, PFRES_SYNPROXY);
return (PF_SYNPROXY_DROP);
}
+ }
+
+ if (((th->th_flags & (TH_SYN|TH_ACK)) == TH_SYN) &&
+ dst->state >= TCPS_FIN_WAIT_2 &&
+ src->state >= TCPS_FIN_WAIT_2) {
+ if (pf_status.debug >= PF_DEBUG_MISC) {
+ printf("pf: state reuse ");
+ pf_print_state(*state);
+ pf_print_flags(th->th_flags);
+ printf("\n");
+ }
+ /* XXX make sure it's the same direction ?? */
+ (*state)->src.state = (*state)->dst.state = TCPS_CLOSED;
+ pf_unlink_state(*state);
+ *state = NULL;
+ return (PF_DROP);
}
if (src->wscale && dst->wscale && !(th->th_flags & TH_SYN)) {
The problem I'm having is that I get intermittent connection
refused/operation not permitted to another machine on the local network.
When I do pfctl -s info I see *huge* numbers of state mismatches:
Status: Enabled for 94 days 01:27:40 Debug: Urgent
State Table Total Rate
current entries 398
searches 986228319 121.4/s
inserts 104049508 12.8/s
removals 104049110 12.8/s
Counters
match 107482262 13.2/s
bad-offset 0 0.0/s
fragment 0 0.0/s
short 0 0.0/s
normalize 42 0.0/s
memory 3125235 0.4/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 0 0.0/s
proto-cksum 13919 0.0/s
state-mismatch 3039814 0.4/s
state-insert 0 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
This is causing serious problems at them moment. It seems that the state
problems occur in certain small time windows (my nagios starts reporting
that every service is connection refused/operation not permitted, which
is about 20 services). Then I get 20 recovery messages.
The firewall rules are trivially simple, $ext_if has 2 ips and $int_if
has one:
interfaces = "{" $ext_if "," $int_if "}"
scrub in all
set skip on lo0
antispoof for $interfaces inet
block out log quick on $ext_if from !$ext_ip1 to any
block in quick on $ext_if from any to 255.255.255.255
block log all
pass in quick inet proto icmp all icmp-type $icmp_types
pass in quick on $int_if from $int_net to any
pass out quick on $int_if from any to $int_net
pass out on $ext_if proto tcp all
pass out on $ext_if proto { udp, icmp } all
pass in on $ext_if proto tcp from any to $ext_ip1 port $tcp_services1
pass in on $ext_if proto tcp from any to $ext_ip2 port $tcp_services2
Does anybody have any idea what's going on and where I can look? This is
a production server so it's seriously influencing the quality of the
hosted services. :-(
Regards,
Sebastiaan
--------------ms030107090002090101050409
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJUTCC
AwMwggJsoAMCAQICEFN8DarMNuuKJDEtfs0UaqUwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDYzMDEzNTE1N1oX
DTA5MDYzMDEzNTE1N1owaDEQMA4GA1UEBBMHdmFuIEVyazETMBEGA1UEKhMKU2ViYXN0aWFh
bjEbMBkGA1UEAxMSU2ViYXN0aWFhbiB2YW4gRXJrMSIwIAYJKoZIhvcNAQkBFhNzZWJzdGVy
QHNlYnN0ZXIuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsJDDAeYHVmH/
GVxi+bhFx27dmg++9BdhPJfk8k041sqEqq7oXnR2GT54quY3Ac7A1BuOM2JvoICraGmjud4y
b3EanRnqGIK6iH+VAhhTlV/Owrb2Qm1e13DLxwLp1SocSQl4IrEbF9Y5H3ASdIrE0iFqkpju
nPiiHeNhz3LaI5ipjiluKYoH+F6gPx8njHoaDxPePCkSLg4r0IA0afLM74LVZxCRBZEfyRZS
J6VVUJefKlz91dWSzR/3xSw/rO4u9Ds/Zh7VBUKy3K+YFryHxRpUek0gSepE1b70Q39L9Sqd
M/NZqMvFpwrqgW2Zh2Nh8nqRge90maR4ypBzz3GzLwIDAQABozAwLjAeBgNVHREEFzAVgRNz
ZWJzdGVyQHNlYnN0ZXIuY29tMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEAS1Sk
NMgDVzb0ktO9tPPacV0KdKhTYOHcICVmuDEe2sFHOkjLAI1iAKp640pqJEVqvRnfRcCFJ9hK
koPjjVZ+ui2rVmJWBG6FSloLRS/YYED4tUAw6DQhK61UOpjkpQxjCdm+5bHG/2ZgJAda1j0x
uiN822+xFkcaW/5PQgxSRxcwggMDMIICbKADAgECAhBTfA2qzDbriiQxLX7NFGqlMA0GCSqG
SIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo
UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBD
QTAeFw0wODA2MzAxMzUxNTdaFw0wOTA2MzAxMzUxNTdaMGgxEDAOBgNVBAQTB3ZhbiBFcmsx
EzARBgNVBCoTClNlYmFzdGlhYW4xGzAZBgNVBAMTElNlYmFzdGlhYW4gdmFuIEVyazEiMCAG
CSqGSIb3DQEJARYTc2Vic3RlckBzZWJzdGVyLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEP
ADCCAQoCggEBALCQwwHmB1Zh/xlcYvm4Rcdu3ZoPvvQXYTyX5PJNONbKhKqu6F50dhk+eKrm
NwHOwNQbjjNib6CAq2hpo7neMm9xGp0Z6hiCuoh/lQIYU5VfzsK29kJtXtdwy8cC6dUqHEkJ
eCKxGxfWOR9wEnSKxNIhapKY7pz4oh3jYc9y2iOYqY4pbimKB/heoD8fJ4x6Gg8T3jwpEi4O
K9CANGnyzO+C1WcQkQWRH8kWUielVVCXnypc/dXVks0f98UsP6zuLvQ7P2Ye1QVCstyvmBa8
h8UaVHpNIEnqRNW+9EN/S/UqnTPzWajLxacK6oFtmYdjYfJ6kYHvdJmkeMqQc89xsy8CAwEA
AaMwMC4wHgYDVR0RBBcwFYETc2Vic3RlckBzZWJzdGVyLmNvbTAMBgNVHRMBAf8EAjAAMA0G
CSqGSIb3DQEBBQUAA4GBAEtUpDTIA1c29JLTvbTz2nFdCnSoU2Dh3CAlZrgxHtrBRzpIywCN
YgCqeuNKaiRFar0Z30XAhSfYSpKD441Wfrotq1ZiVgRuhUpaC0Uv2GBA+LVAMOg0ISutVDqY
5KUMYwnZvuWxxv9mYCQHWtY9MbojfNtvsRZHGlv+T0IMUkcXMIIDPzCCAqigAwIBAgIBDTAN
BgkqhkiG9w0BAQUFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTES
MBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UE
CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBl
cnNvbmFsIEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0
aGF3dGUuY29tMB4XDTAzMDcxNzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMC
WkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1Ro
YXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GN
ADCBiQKBgQDEpjxVc1X7TrnKmVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAK
MNcCY1osiRVwjt3J8CuFWqo/cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTX
p6a7n2XRxSpUhQ9IBH+nttE8YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYB
Af8CAQAwQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBl
cnNvbmFsRnJlZW1haWxDQS5jcmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYD
VQQDExFQcml2YXRlTGFiZWwyLTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2as
Zw9/r6y+whehQ5aUnX9MIbj4Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSe
JVCUYsfbJ3FXJY3dqZw5jowgT2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHT
HUb/XV9lTzGCA3EwggNtAgEBMHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBD
b25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp
bCBJc3N1aW5nIENBAhBTfA2qzDbriiQxLX7NFGqlMAkGBSsOAwIaBQCgggHQMBgGCSqGSIb3
DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA5MDMyNTA4NTk0M1owIwYJKoZI
hvcNAQkEMRYEFGsiiFdvEXGmJFJ5WvjOttxTZqRDMF8GCSqGSIb3DQEJDzFSMFAwCwYJYIZI
AWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUr
DgMCBzANBggqhkiG9w0DAgIBKDCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTEl
MCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3Rl
IFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEFN8DarMNuuKJDEtfs0UaqUwgYcGCyqG
SIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRp
bmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vp
bmcgQ0ECEFN8DarMNuuKJDEtfs0UaqUwDQYJKoZIhvcNAQEBBQAEggEAILsayitdUh9jm+tv
hc0UqLlT9lm5q8wydEZeDatDlTbOeC6mypqbVVVjj4CgSkvxgRopDzj3GTmNHcTvSOdms1Na
Q+zV7ILniwkscSM3S4aS80kiznA3z4QM50KdSlwQMRcP9xCXhQ5SinvTlFrD0KzfrwzMjP99
RcMJOm+dbAhrIEeCf37/YkVgIEvBN6O3sidzmji0MGkIEbCSxCqefoavu0fIs1/uKkWQt7zw
wfC9GyYaxuDf7O/L62cPlom6Uc7BEzoysxA2Q6RwbZL4/uvOR+00Wmy9tCq99l8YvWP3T6Hm
37A/0mnNPmaRtYWgpfPz3Fxt/cltzsX61/2CpgAAAAAAAA==
--------------ms030107090002090101050409--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49C9F27F.3010505>