Date: Fri, 22 Jan 2010 09:08:45 +0100 From: Florian Smeets <flo@smeets.im> To: John Baldwin <jhb@freebsd.org> Cc: Luigi Rizzo <luigi@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: 7.2-STABLE page fault with kernel from 12.01.2010 / crashinfo available Message-ID: <4B595D0D.3060904@smeets.im> In-Reply-To: <201001211515.32562.jhb@freebsd.org> References: <4B58280C.50602@smeets.im> <201001211405.35615.jhb@freebsd.org> <4B58A66E.3040900@smeets.im> <201001211515.32562.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 1/21/10 9:15 PM, John Baldwin wrote:
> On Thursday 21 January 2010 2:09:34 pm Florian Smeets wrote:
>> On 1/21/10 8:05 PM, John Baldwin wrote:
>>> On Thursday 21 January 2010 1:33:35 pm Florian Smeets wrote:
>>>> On 1/21/10 6:58 PM, John Baldwin wrote:
>>>>> On Thursday 21 January 2010 8:25:22 am Florian Smeets wrote:
>>>>>> (kgdb) frame 8
>>>>>> #8 0xc05f8b28 in ip_forward (m=0xc23dc900, srcrt=0) at
>>>>>> /usr/src/sys/netinet/ip_input.c:1307
>>>>>> 1307 m_copydata(m, 0, mcopy->m_len, mtod(mcopy, caddr_t));
>>>>>> (kgdb) l
>>>>>> 1302 mcopy = NULL;
>>>>>> 1303 }
>>>>>> 1304 if (mcopy != NULL) {
>>>>>> 1305 mcopy->m_len = min(ip->ip_len, M_TRAILINGSPACE(mcopy));
>>>>>> 1306 mcopy->m_pkthdr.len = mcopy->m_len;
>>>>>> 1307 m_copydata(m, 0, mcopy->m_len, mtod(mcopy, caddr_t));
>>>>>> 1308 }
>>>>>> 1309
>>>>>> 1310 #ifdef IPSTEALTH
>>>>>> 1311 if (!ipstealth) {
>>>>>> (kgdb) p *m
>>>>>> $1 = {m_hdr = {mh_next = 0x0, mh_nextpkt = 0x0, mh_data = 0xc271e80e
>>>>>> "E\020", mh_len = 164, mh_flags = 3, mh_type = 1, pad = "\000"}, M_dat
> =
>>>>>> {MH = {MH_pkthdr = {rcvif = 0xc20a4800, header = 0x0, len = 164,
>>>>>> csum_flags = 3072,
>>>>>> csum_data = 65535, tso_segsz = 0, ether_vtag = 0, tags =
>>>>>> {slh_first = 0xc35bc380}}, MH_dat = {MH_ext = {ext_buf = 0xc271e800 "",
>>>>>> ext_free = 0, ext_args = 0x0, ext_size = 2048, ref_cnt = 0xc2703ab4,
>>>>>> ext_type = 6},
>>>>>> MH_databuf =
>>>>>> "\000?q?\000\000\000\000\000\000\000\000\000\b\000\000?:p?
>>>>> \006\000\000\000dL?\t<+?\202\200\020
>>>>>> O/\207\000\000\001\001\b\n-?b\230qms?\000\000\004\001?l?
> \000\000\001%r???
>>>>> \200\000????\034?Ot?\b?{sr\000\034org.jboss.mq.ConnectionToken?\b&?
>>>>>
>>>
> \237N\002\000\005I\000\004hashZ\000\asameJVML\000\bclientIDt\000\022Ljava/l\000\220\032Ae\207\000\002?
>>>>> 36@\210d\021\000\001?
>>> \001B\000!E\000\001@bV\000\000@2\032$W\213\n\034"...}},
>>>>>>
>>>>>> M_databuf =
>>>>>> "\000H\n?\000\000\000\000?\000\000\000\000\f\000\000??
>>>>> \000\000\000\000\000\000\200?[?\000?q?
>>>>> \000\000\000\000\000\000\000\000\000\b\000\000?:p?\006\000\000\000dL?
> \t<+?
>>>>> \202\200\020
>>>>>> O/\207\000\000\001\001\b\n-?b\230qms?\000\000\004\001?l?
> \000\000\001%r???
>>>>> \200\000????\034?Ot?\b?{sr\000\034org.jboss.mq.ConnectionToken?\b&?
>>>>>
>>>
> \237N\002\000\005I\000\004hashZ\000\asameJVML\000\bclientIDt\000\022Ljava/l\000\220\032Ae\207\000\002?
>>>>> 3"...}}
>>>>>
>>>>> Ok, can you do 'p *m_copy'?
>>>>>
>>>>
>>>> What ever you want :-)
>>>>
>>>> (kgdb) p *m_copy
>>>> No symbol "m_copy" in current context.
>>>> (kgdb) p *m_copydata
>>>> $2 = {void (const struct mbuf *, int, int, caddr_t)}
> 0xc0572e10<m_copydata>
>>>> (kgdb) p *mcopy
>>>> $1 = {m_hdr = {mh_next = 0x0, mh_nextpkt = 0x0, mh_data = 0xc23cce34
>>>> "E\020", mh_len = 204, mh_flags = 2, mh_type = 1, pad = "\000"}, M_dat =
>>>> {MH = {MH_pkthdr = {rcvif = 0xc20a4800, header = 0x0,
>>>> len = 204, csum_flags = 3072, csum_data = 65535, tso_segsz = 0,
>>>> ether_vtag = 0, tags = {slh_first = 0xc23c3e00}}, MH_dat = {MH_ext =
>>>> {ext_buf = 0x84001045<Address 0x84001045 out of bounds>,
>>>
>>> Hmm, ok. Can you do 'p *ip'? mcopy->m_len (204) is larger than m->m_len
>>> (164). That shouldn't be the case unless ip->ip_len is somehow larger
> than m-
>>>> m_len.
>>>
>>
>> (kgdb) p *ip
>> $3 = {ip_hl = 5, ip_v = 4, ip_tos = 16 '\020', ip_len = 33792, ip_id =
>> 61492, ip_off = 64, ip_ttl = 64 '@', ip_p = 6 '\006', ip_sum = 34849,
>> ip_src = {s_addr = 355576000}, ip_dst = {
>> s_addr = 2251401408}}
>
> Looks like ip_len is in network byte order instead of host byte order and that
> is causing the problem. 33792 == 0x8400. Swapping that gives 0x84 == 132
> which would be a reasonable length. Are you using any firewall rules that
> would rewrite packets? I wonder if you are having a packet rewritten and the
> new IP header is written in network byte order, but we swap the IP header len
> field to host byte order earlier in ip_input(). Luigi Rizzo may have some
> insight into this.
>
Well, when looking at MH_databuf i see Jboss MQ traffic that would mean
that this traffic was coming from or going to an IPsec tunnel, i could
say for sure when i would have a clue how to get an IP address from
something like ip_src = {s_addr = 355576000}.
If it really is IPsec traffic then there are no rewrite rules only 10 pf
pass rules on the enc0 interface and a "scrub in all" rule.
Perhaps it matters that i have these set:
net.enc.out.ipsec_bpf_mask=0x00000001
net.enc.out.ipsec_filter_mask=0x00000001
net.enc.in.ipsec_bpf_mask=0x00000002
net.enc.in.ipsec_filter_mask=0x00000002
so that i can filter the "encapsulated" traffic.
Thanks,
Florian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4B595D0D.3060904>