Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 04 Jul 2011 10:08:45 +1000
From:      "Peter Ross" <Peter.Ross@bogen.in-berlin.de>
To:        "Scott Sipe" <cscotts@gmail.com>
Cc:        freebsd-stable List <freebsd-stable@freebsd.org>, Jeremy Chadwick <freebsd@jdc.parodius.com>
Subject:   Re: scp: Write Failed: Cannot allocate memory
Message-ID:  <20110704100845.94513n3znbabpthp@webmail.in-berlin.de>
In-Reply-To: <54D65EC5-9A9B-4F96-BB45-1904F2147CBA@gmail.com>
References:  <BANLkTinGV6wBvVGyA2PjZ9fnvYt5hKsLOA@mail.gmail.com> <20110701222232.GA33935@icarus.home.lan> <20110702045435.GA81502@DataIX.net> <54D65EC5-9A9B-4F96-BB45-1904F2147CBA@gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi all,

I noticed a similar problem last week. It is also very similar to one =20
reported last year:

http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058708.html

My server is a Dell T410 server with the same bge card (the same =20
pciconf -lvc output as described by Mahlon:

http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058711.html

Yours, Scott, is a em(4)..

Another similarity: In all cases we are using VirtualBox. I just want =20
to mention it, in case it matters. I am still running VirtualBox 3.2.

Most of the time kstat.zfs.misc.arcstats.size was reaching =20
vfs.zfs.arc_max then, but I could catch one or two cases then the =20
value was still below.

I added vfs.zfs.prefetch_disable=3D1 to sysctl.conf but it does not help.

BTW: It looks as ARC only gives back the memory when I destroy the ZFS =20
(a cloned snapshot containing virtual machines). Even if nothing =20
happens for hours the buffer isn't released..

My machine was still running 8.2-PRERELEASE so I am upgrading.

I am happy to give information gathered on old/new kernel if it helps.

Regards
Peter

Quoting "Scott Sipe" <cscotts@gmail.com>:

>
> On Jul 2, 2011, at 12:54 AM, jhell wrote:
>
>> On Fri, Jul 01, 2011 at 03:22:32PM -0700, Jeremy Chadwick wrote:
>>> On Fri, Jul 01, 2011 at 03:13:17PM -0400, Scott Sipe wrote:
>>>> I'm running 8.2-RELEASE and am having new problems with scp. When scpin=
g
>>>> files to a ZFS directory on the FreeBSD server -- most notably large fi=
les
>>>> -- the transfer frequently dies after just a few seconds. In my =20
>>>> last test, I
>>>> tried to scp an 800mb file to the FreeBSD system and the transfer =20
>>>> died after
>>>> 200mb. It completely copied the next 4 times I tried, and then =20
>>>> died again on
>>>> the next attempt.
>>>>
>>>> On the client side:
>>>>
>>>> "Connection to home closed by remote host.
>>>> lost connection"
>>>>
>>>> In /var/log/auth.log:
>>>>
>>>> Jul  1 14:54:42 freebsd sshd[18955]: fatal: Write failed: Cannot alloca=
te
>>>> memory
>>>>
>>>> I've never seen this before and have used scp before to transfer =20
>>>> large files
>>>> without problems. This computer has been used in production for months =
and
>>>> has a current uptime of 36 days. I have not been able to notice =20
>>>> any problems
>>>> copying files to the server via samba or netatalk, or any problems in
>>>> apache.
>>>>
>>>> Uname:
>>>>
>>>> FreeBSD xeon 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Sat Feb 19 01:02:54 ES=
T
>>>> 2011     root@xeon:/usr/obj/usr/src/sys/GENERIC  amd64
>>>>
>>>> I've attached my dmesg and output of vmstat -z.
>>>>
>>>> I have not restarted the sshd daemon or rebooted the computer.
>>>>
>>>> Am glad to provide any other information or test anything else.
>>>>
>>>> {snip vmstat -z and dmesg}
>>>
>>> You didn't provide details about your networking setup (rc.conf,
>>> ifconfig -a, etc.).  netstat -m would be useful too.
>>>
>>> Next, please see this thread circa September 2010, titled "Network
>>> memory allocation failures":
>>>
>>> http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/thread.=
html#58708
>>>
>>> The user in that thread is using rsync, which relies on scp by default.
>>> I believe this problem is similar, if not identical, to yours.
>>>
>>
>> Please also provide your output of ( /usr/bin/limits -a ) for the server
>> end and the client.
>>
>> I am not quite sure I agree with the need for ifconfig -a but some
>> information about the networking driver your using for the interface
>> would be helpful, uptime of the boxes. And configuration of the pool.
>> e.g. ( zpool status -a ;zfs get all <poolname> ) You should probably
>> prop this information up somewhere so you can reference by URL whenever
>> needed.
>>
>> rsync(1) does not rely on scp(1) whatsoever but rsync(1) can be made to
>> use ssh(1) instead of rsh(1) and I believe that is what Jeremy is
>> stating here but correct me if I am wrong. It does use ssh(1) by
>> default.
>>
>> Its a possiblity as well that if using tmpfs(5) or mdmfs(8) for /tmp
>> type filesystems that rsync(1) may be just filling up your temp ram area
>> and causing the connection abort which would be expected. ( df -h ) would
>> help here.
>
> Hello,
>
> I'm not using tmpfs/mdmfs at all. The clients yesterday were 3 =20
> different OSX computers (over gigabit). The FreeBSD server has 12gb =20
> of ram and no bce adapter. For what it's worth, the server is backed =20
> up remotely every night with rsync (remote FreeBSD uses rsync to =20
> pull) to an offsite (slow cable connection) FreeBSD computer, and I =20
> have not seen any errors in the nightly rsync.
>
> Sorry for the omission of networking info, here's the output of the =20
> requested commands and some that popped up in the other thread:
>
> http://www.cap-press.com/misc/
>
> In rc.conf:  ifconfig_em1=3D"inet 10.1.1.1 netmask 255.255.0.0"
>
> Scott
>
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
>





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110704100845.94513n3znbabpthp>