Date: Fri, 13 Feb 2015 12:46:22 +1300 From: Andrew Thompson <thompsa@FreeBSD.org> To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com> Cc: freebsd-xen@freebsd.org Subject: Re: xenstore memory issue Message-ID: <CAFAOGNS6w1uy6OUOXEohZH29b8X07rZc83%2B6O7fUA=SDn1VveQ@mail.gmail.com> In-Reply-To: <54DA303F.9020203@citrix.com> References: <CAFAOGNTCJsXr0VOUHAtFoPvQO5VWSUvL_VKkxYoaCtrZdC6zUg@mail.gmail.com> <54D88BD5.7050703@citrix.com> <CAFAOGNTqMOF=wK79PzoOV8d8Qh-4CkHPs4rxy9gmfBEbQ9-CfA@mail.gmail.com> <CAFAOGNR6u3O9WhG0tjey-7thKTM8HXweK2vn4TJRLvE7OezF%2BA@mail.gmail.com> <54DA303F.9020203@citrix.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11 February 2015 at 05:22, Roger Pau Monn=C3=A9 <roger.pau@citrix.com> w= rote: > Hello Andrew, > > El 10/02/15 a les 10.22, Andrew Thompson ha escrit: > > On 10 February 2015 at 16:25, Andrew Thompson <thompsa@freebsd.org> > wrote: > > > >> On 9 February 2015 at 23:28, Roger Pau Monn=C3=A9 <roger.pau@citrix.co= m> > wrote: > >> A bit of trial and error with dtrace has narrowed this down. I can cau= se > >> the leak by just opening /dev/xen/xenstore > > Thanks for doing all this legwork! IMHO the best solution is to switch > xenstore dev to use cdevpriv in order to store each client data. What we > are doing right now (storing client data in dev->si_dvr1) is plain > wrong. I've uploaded two patches (one for HEAD and one for stable/10) so > that you can try it also, please report back whether this fixes your > problem or not: > > https://people.freebsd.org/~royger/xenstore_fix/ > > I have tested this on 10.0 and it does fix the issue. After 36000 allocations there are just 6 active at 13K xenstore 6 13K - 36699 16,32,64,128,256,512 I looked in to why this only affected one of my many VMs as they run xe-update-guest-attrs and nova-agent which should have triggered the leak. They are all stuck on xsdread when reading from the xenstore so the update loop has stopped. UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 13250 13249 0 20 0 16552 1968 xsdread S - 5:57.60 /usr/local/bin/xenstore-read domid (xenstore) Is there an issue with the xs_dev_read and xs_queue_reply logic or would the Xen host be timing out? regards, Andrew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFAOGNS6w1uy6OUOXEohZH29b8X07rZc83%2B6O7fUA=SDn1VveQ>