From owner-freebsd-isp Sun Oct 7 13: 6:45 2001 Delivered-To: freebsd-isp@freebsd.org Received: from femail4.sdc1.sfba.home.com (femail4.sdc1.sfba.home.com [24.0.95.84]) by hub.freebsd.org (Postfix) with ESMTP id 8DA3037B401 for ; Sun, 7 Oct 2001 13:06:41 -0700 (PDT) Received: from veager.jwweeks.com ([65.14.122.116]) by femail4.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011007200640.HRNA2835.femail4.sdc1.sfba.home.com@veager.jwweeks.com> for ; Sun, 7 Oct 2001 13:06:40 -0700 Date: Sun, 7 Oct 2001 16:06:32 -0400 (EDT) From: Jim Weeks X-Sender: jim@veager.jwweeks.com To: freebsd-isp@freebsd.org Subject: Webalizer oddness Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Looking for another head on this one, because I am totally confused. After cleaning up httpd.conf and -HUPing httpd on one of my machines, I noticed that the once a day run of webalizer isn't picking up on any traffic logged after Oct, 4th. After examining the log files, I can see that there hasn't been any gaps in logging, neither have I change the format of the log entries. Here is where the oddness comes in. If I copy one of the log files and delete everything before the 5th, I can do a test run which accurately shows all traffic for the 5th through 7th. If I delete dividual days, that works too. If the log is run in its entirety, everything after the 4th is ignored. Any ideas would be appreciated, I'm tapped out, -- Jim Weeks To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 13:26:10 2001 Delivered-To: freebsd-isp@freebsd.org Received: from sv07e.atm-tzs.kmjeuro.com (sv07e.atm-tzs.kmjeuro.com [193.81.94.207]) by hub.freebsd.org (Postfix) with ESMTP id 176CB37B401 for ; Sun, 7 Oct 2001 13:26:05 -0700 (PDT) Received: (from root@localhost) by sv07e.atm-tzs.kmjeuro.com (8.11.5/8.11.4) id f97KQ0392819 for freebsd-isp@freebsd.org; Sun, 7 Oct 2001 22:26:00 +0200 (CEST) (envelope-from k.joch@kmjeuro.com) Received: from karl (787fd3d9e9d5906a9f99dd912b091b6a@adsl.ooe.kmjeuro.com [193.154.186.21]) (authenticated) by sv07e.atm-tzs.kmjeuro.com (8.11.5/8.11.4) with ESMTP id f97KPoV92550; Sun, 7 Oct 2001 22:25:50 +0200 (CEST) (envelope-from k.joch@kmjeuro.com) Message-ID: <04e301c14f6e$c0f19820$0a05a8c0@ooe.kmjeuro.com> From: "Karl M. Joch" To: "Jim Weeks" , References: Subject: Re: Webalizer oddness Date: Sun, 7 Oct 2001 22:29:08 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X--virus-scanner: scanned for Virus and dangerous attachments on sv07e.atm-tzs.kmjeuro.com (System Setup/Maintainance: http://www.ctseuro.com/) Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Have the same problem here. also installing the actual version doesnt help (says cant recover last runs data). so i temporary removed the entries from the 1st to the 4th and continued logs with the 5th. keeping the logs and will rerun october when a solution is found. something in the .current file must be screwed up with this dates because i needed to delete the .current file and the 10/2001 entry from .history. Karl -- -- Best regards / Mit freundlichen Gruessen, Karl M. Joch KMJ Consulting - CTS Consulting & Trade Service http://www.kmjeuro.com - http://www.ctseuro.com k.joch@kmjeuro.com - k.joch@ctseuro.com GSM : +43-664-3407888 Unsere Services: http://www.proline.at - Netzwerk und Sicherheitstechnik http://www.eushop.net - Onlineshop und Applikationen einfach mieten http://www.freebsd.at - Power Operating System ----- Original Message ----- From: "Jim Weeks" To: Sent: Sunday, October 07, 2001 10:06 PM Subject: Webalizer oddness > Looking for another head on this one, because I am totally confused. > > After cleaning up httpd.conf and -HUPing httpd on one of my machines, I > noticed that the once a day run of webalizer isn't picking up on any > traffic logged after Oct, 4th. After examining the log files, I can see > that there hasn't been any gaps in logging, neither have I change the > format of the log entries. > > Here is where the oddness comes in. If I copy one of the log files and > delete everything before the 5th, I can do a test run which accurately shows > all traffic for the 5th through 7th. If I delete dividual days, that > works too. If the log is run in its entirety, everything after the 4th is > ignored. > > Any ideas would be appreciated, I'm tapped out, > > -- > Jim Weeks > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 13:35: 7 2001 Delivered-To: freebsd-isp@freebsd.org Received: from femail4.sdc1.sfba.home.com (femail4.sdc1.sfba.home.com [24.0.95.84]) by hub.freebsd.org (Postfix) with ESMTP id 8CDD337B407 for ; Sun, 7 Oct 2001 13:35:02 -0700 (PDT) Received: from veager.jwweeks.com ([65.14.122.116]) by femail4.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011007203502.IGZU2835.femail4.sdc1.sfba.home.com@veager.jwweeks.com>; Sun, 7 Oct 2001 13:35:02 -0700 Date: Sun, 7 Oct 2001 16:35:01 -0400 (EDT) From: Jim Weeks X-Sender: jim@veager.jwweeks.com To: "Karl M. Joch" Cc: freebsd-isp@FreeBSD.ORG Subject: Re: Webalizer oddness In-Reply-To: <04e301c14f6e$c0f19820$0a05a8c0@ooe.kmjeuro.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Karl, Thanks for your prompt reply. I thought I might be going crazy. I assume a lot of people will begin to realize that there is a problem. If I come up with something I will let you know. -- Jim Weeks On Sun, 7 Oct 2001, Karl M. Joch wrote: > Have the same problem here. also installing the actual version doesnt help > (says cant recover last runs data). so i temporary removed the entries from > the 1st to the 4th and continued logs with the 5th. keeping the logs and > will rerun october when a solution is found. something in the .current file > must be screwed up with this dates because i needed to delete the .current > file and the 10/2001 entry from .history. > > Karl > > > -- > -- > Best regards / Mit freundlichen Gruessen, > > Karl M. Joch > KMJ Consulting - CTS Consulting & Trade Service > http://www.kmjeuro.com - http://www.ctseuro.com > k.joch@kmjeuro.com - k.joch@ctseuro.com > > GSM : +43-664-3407888 > > Unsere Services: > http://www.proline.at - Netzwerk und Sicherheitstechnik > http://www.eushop.net - Onlineshop und Applikationen einfach mieten > http://www.freebsd.at - Power Operating System > ----- Original Message ----- > From: "Jim Weeks" > To: > Sent: Sunday, October 07, 2001 10:06 PM > Subject: Webalizer oddness > > > > Looking for another head on this one, because I am totally confused. > > > > After cleaning up httpd.conf and -HUPing httpd on one of my machines, I > > noticed that the once a day run of webalizer isn't picking up on any > > traffic logged after Oct, 4th. After examining the log files, I can see > > that there hasn't been any gaps in logging, neither have I change the > > format of the log entries. > > > > Here is where the oddness comes in. If I copy one of the log files and > > delete everything before the 5th, I can do a test run which accurately > shows > > all traffic for the 5th through 7th. If I delete dividual days, that > > works too. If the log is run in its entirety, everything after the 4th is > > ignored. > > > > Any ideas would be appreciated, I'm tapped out, > > > > -- > > Jim Weeks > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-isp" in the body of the message > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 15: 7:36 2001 Delivered-To: freebsd-isp@freebsd.org Received: from velvet.sensation.net.au (serial1-2-velvet-brunswick.sensation.net.au [203.20.114.195]) by hub.freebsd.org (Postfix) with ESMTP id C439237B403 for ; Sun, 7 Oct 2001 15:07:31 -0700 (PDT) Received: from localhost (rowan@localhost) by velvet.sensation.net.au (8.9.3/8.9.3) with ESMTP id IAA45456 for ; Mon, 8 Oct 2001 08:07:28 +1000 (EST) (envelope-from rowan@sensation.net.au) X-Authentication-Warning: velvet.sensation.net.au: rowan owned process doing -bs Date: Mon, 8 Oct 2001 08:07:27 +1000 (EST) From: Rowan Crowe To: freebsd-isp@freebsd.org Subject: Re: Webalizer oddness In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Sun, 7 Oct 2001, Jim Weeks wrote: > Looking for another head on this one, because I am totally confused. > > After cleaning up httpd.conf and -HUPing httpd on one of my machines, I > noticed that the once a day run of webalizer isn't picking up on any > traffic logged after Oct, 4th. After examining the log files, I can see > that there hasn't been any gaps in logging, neither have I change the > format of the log entries. > > Here is where the oddness comes in. If I copy one of the log files and > delete everything before the 5th, I can do a test run which accurately shows > all traffic for the 5th through 7th. If I delete dividual days, that > works too. If the log is run in its entirety, everything after the 4th is > ignored. Hi Jim, How big are the log files? Any chance they're over 2Gb (2^31)? I've seen some strange things happen (at the application level) with log files over this size, presumably due to the use of a signed long. Your mention of deleting previous days made me wonder. Cheers. -- Rowan Crowe camrecord.com / camdiscover.com / Sensation Internet Services Melbourne, Australia To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 15:12:39 2001 Delivered-To: freebsd-isp@freebsd.org Received: from anjin.xpress.se (anjin.xpress.se [195.162.64.2]) by hub.freebsd.org (Postfix) with ESMTP id 0FD2637B405 for ; Sun, 7 Oct 2001 15:12:36 -0700 (PDT) Received: (from apache@localhost) by anjin.xpress.se (8.11.0/8.11.1) id f97M9Lp14111 for freebsd-isp@freebsd.org; Mon, 8 Oct 2001 00:09:21 +0200 X-Authentication-Warning: anjin.xpress.se: apache set sender to rickard@xpress.se using -f To: freebsd-isp@freebsd.org Subject: Dummynet & traffic-shaping Message-ID: <1002492561.3bc0d2916b700@webmail.xpress.se> Date: Mon, 08 Oct 2001 00:09:21 +0200 (CEST) From: =?ISO-8859-1?Q?Rickard_Svor=E9n?= MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: IMP/PHP IMAP webmail program Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Greetings Am setting up an freebsd machine to do traffic-shaping on some customers. The situation is following: Am feeding a site with 4mbps of internetcapacity - Have approx 230 user incoming via catv - Approx 100 user attached directly via Ethernet Hardware = Dual PPro 200Mhz, 128 RAM, 1 Intel EtherExpress NIC & 1 Realtek 8139 The catv customers should have 3.5 mbps and the rest should have 500kbps. Have tested dummynet with my workstation and pico bsd, it worked well. But what considerations should be taken regarding hardware and queue sizes when over 300 users are flowing through the bridge? What should be considered as "normal". Trial and error is an option of course but this would generate a lot of angry phonecalls to me ;-) TIA! -Rickard To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 15:28:53 2001 Delivered-To: freebsd-isp@freebsd.org Received: from sv07e.atm-tzs.kmjeuro.com (sv07e.atm-tzs.kmjeuro.com [193.81.94.207]) by hub.freebsd.org (Postfix) with ESMTP id 7039B37B409 for ; Sun, 7 Oct 2001 15:28:47 -0700 (PDT) Received: (from root@localhost) by sv07e.atm-tzs.kmjeuro.com (8.11.5/8.11.4) id f97MSj196069 for freebsd-isp@freebsd.org; Mon, 8 Oct 2001 00:28:45 +0200 (CEST) (envelope-from k.joch@kmjeuro.com) Received: from karl (398b8dbe831824c5f030912a5b881f0a@adsl.ooe.kmjeuro.com [193.154.186.21]) (authenticated) by sv07e.atm-tzs.kmjeuro.com (8.11.5/8.11.4) with ESMTP id f97MSaV95800; Mon, 8 Oct 2001 00:28:36 +0200 (CEST) (envelope-from k.joch@kmjeuro.com) Message-ID: <001a01c14f7f$e7587720$0a05a8c0@ooe.kmjeuro.com> From: "Karl M. Joch" To: "Rowan Crowe" , References: Subject: Re: Webalizer oddness Date: Mon, 8 Oct 2001 00:32:04 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X--virus-scanner: scanned for Virus and dangerous attachments on sv07e.atm-tzs.kmjeuro.com (System Setup/Maintainance: http://www.ctseuro.com/) Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org ----- Original Message ----- From: "Rowan Crowe" To: Sent: Monday, October 08, 2001 12:07 AM Subject: Re: Webalizer oddness > On Sun, 7 Oct 2001, Jim Weeks wrote: > > > Looking for another head on this one, because I am totally confused. > > > > After cleaning up httpd.conf and -HUPing httpd on one of my machines, I > > noticed that the once a day run of webalizer isn't picking up on any > > traffic logged after Oct, 4th. After examining the log files, I can see > > that there hasn't been any gaps in logging, neither have I change the > > format of the log entries. > > > > Here is where the oddness comes in. If I copy one of the log files and > > delete everything before the 5th, I can do a test run which accurately shows > > all traffic for the 5th through 7th. If I delete dividual days, that > > works too. If the log is run in its entirety, everything after the 4th is > > ignored. > > Hi Jim, > > How big are the log files? Any chance they're over 2Gb (2^31)? I've seen > some strange things happen (at the application level) with log files over > this size, presumably due to the use of a signed long. Your mention of > deleting previous days made me wonder. > > Cheers. > The size of the log file is not the error in this case. i made a script which runs (since a long time) daily all of the virtual servers. the logfiles then are compressed and archived. so it is always only one day. the smallest of this daily logs where it happens has about 200K, the biggest about 70MB. it looks like webalizer is sceewed up with the .current file. Karl > > -- > Rowan Crowe > camrecord.com / camdiscover.com / Sensation Internet Services > Melbourne, Australia > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 15:40:18 2001 Delivered-To: freebsd-isp@freebsd.org Received: from femail4.sdc1.sfba.home.com (femail4.sdc1.sfba.home.com [24.0.95.84]) by hub.freebsd.org (Postfix) with ESMTP id DD67137B405 for ; Sun, 7 Oct 2001 15:40:14 -0700 (PDT) Received: from veager.jwweeks.com ([65.14.122.116]) by femail4.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011007224014.LAPX2835.femail4.sdc1.sfba.home.com@veager.jwweeks.com>; Sun, 7 Oct 2001 15:40:14 -0700 Date: Sun, 7 Oct 2001 18:40:13 -0400 (EDT) From: Jim Weeks X-Sender: jim@veager.jwweeks.com To: "Karl M. Joch" Cc: Rowan Crowe , freebsd-isp@FreeBSD.ORG Subject: Re: Webalizer oddness In-Reply-To: <001a01c14f7f$e7587720$0a05a8c0@ooe.kmjeuro.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Rowan, Karl is right. The problem definitely stems from the date translation. I have even compiled test log files with 20 or so entries for the 4th and 5th only. One thing we haven't addressed is the version. I am in the process of upgrading to 2.0-6 from 1.30 on this particular machine. It happens to be 3.5.1-Stable, so there are a few hurdles. -- Jim Weeks On Mon, 8 Oct 2001, Karl M. Joch wrote: > ----- Original Message ----- > From: "Rowan Crowe" > To: > Sent: Monday, October 08, 2001 12:07 AM > Subject: Re: Webalizer oddness > > > > On Sun, 7 Oct 2001, Jim Weeks wrote: > > > > > Looking for another head on this one, because I am totally confused. > > > > > > After cleaning up httpd.conf and -HUPing httpd on one of my machines, I > > > noticed that the once a day run of webalizer isn't picking up on any > > > traffic logged after Oct, 4th. After examining the log files, I can see > > > that there hasn't been any gaps in logging, neither have I change the > > > format of the log entries. > > > > > > Here is where the oddness comes in. If I copy one of the log files and > > > delete everything before the 5th, I can do a test run which accurately > shows > > > all traffic for the 5th through 7th. If I delete dividual days, that > > > works too. If the log is run in its entirety, everything after the 4th > is > > > ignored. > > > > Hi Jim, > > > > How big are the log files? Any chance they're over 2Gb (2^31)? I've seen > > some strange things happen (at the application level) with log files over > > this size, presumably due to the use of a signed long. Your mention of > > deleting previous days made me wonder. > > > > Cheers. > > > > The size of the log file is not the error in this case. i made a script > which runs (since a long time) daily all of the virtual servers. the > logfiles then are compressed and archived. so it is always only one day. the > smallest of this daily logs where it happens has about 200K, the biggest > about 70MB. it looks like webalizer is sceewed up with the .current file. > > Karl > > > > > > > -- > > Rowan Crowe > > camrecord.com / camdiscover.com / Sensation Internet Services > > Melbourne, Australia > > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-isp" in the body of the message > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 15:49:59 2001 Delivered-To: freebsd-isp@freebsd.org Received: from sv07e.atm-tzs.kmjeuro.com (sv07e.atm-tzs.kmjeuro.com [193.81.94.207]) by hub.freebsd.org (Postfix) with ESMTP id 5ADFB37B401 for ; Sun, 7 Oct 2001 15:49:53 -0700 (PDT) Received: (from root@localhost) by sv07e.atm-tzs.kmjeuro.com (8.11.5/8.11.4) id f97MnqV97451 for freebsd-isp@freebsd.org; Mon, 8 Oct 2001 00:49:52 +0200 (CEST) (envelope-from k.joch@kmjeuro.com) Received: from karl (c7f280e8f01f7a8709d8ebfa005f0712@adsl.ooe.kmjeuro.com [193.154.186.21]) (authenticated) by sv07e.atm-tzs.kmjeuro.com (8.11.5/8.11.4) with ESMTP id f97MncV97047; Mon, 8 Oct 2001 00:49:38 +0200 (CEST) (envelope-from k.joch@kmjeuro.com) Message-ID: <001501c14f82$d7bff970$0a05a8c0@ooe.kmjeuro.com> From: "Karl M. Joch" To: "Jim Weeks" Cc: "Rowan Crowe" , References: Subject: Re: Webalizer oddness Date: Mon, 8 Oct 2001 00:52:59 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X--virus-scanner: scanned for Virus and dangerous attachments on sv07e.atm-tzs.kmjeuro.com (System Setup/Maintainance: http://www.ctseuro.com/) Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org ----- Original Message ----- From: "Jim Weeks" To: "Karl M. Joch" Cc: "Rowan Crowe" ; Sent: Monday, October 08, 2001 12:40 AM Subject: Re: Webalizer oddness > Rowan, > > Karl is right. The problem definitely stems from the date translation. I > have even compiled test log files with 20 or so entries for the 4th and > 5th only. > > One thing we haven't addressed is the version. I am in the process of > upgrading to 2.0-6 from 1.30 on this particular machine. It happens to be > 3.5.1-Stable, so there are a few hurdles. > I run 4.3 mostly and 4.4 on some. having 1.30, 2.0.6 and the actual version from the ports 2.1? on a test box. on the actual it writes "cant restore data of last run" (or similiar, sorry already logged out everywhere). i will try the actual port version tomorrow with rebuilding october from the scratch. Karl > -- > Jim Weeks > > > On Mon, 8 Oct 2001, Karl M. Joch wrote: > > > ----- Original Message ----- > > From: "Rowan Crowe" > > To: > > Sent: Monday, October 08, 2001 12:07 AM > > Subject: Re: Webalizer oddness > > > > > > > On Sun, 7 Oct 2001, Jim Weeks wrote: > > > > > > > Looking for another head on this one, because I am totally confused. > > > > > > > > After cleaning up httpd.conf and -HUPing httpd on one of my machines, I > > > > noticed that the once a day run of webalizer isn't picking up on any > > > > traffic logged after Oct, 4th. After examining the log files, I can see > > > > that there hasn't been any gaps in logging, neither have I change the > > > > format of the log entries. > > > > > > > > Here is where the oddness comes in. If I copy one of the log files and > > > > delete everything before the 5th, I can do a test run which accurately > > shows > > > > all traffic for the 5th through 7th. If I delete dividual days, that > > > > works too. If the log is run in its entirety, everything after the 4th > > is > > > > ignored. > > > > > > Hi Jim, > > > > > > How big are the log files? Any chance they're over 2Gb (2^31)? I've seen > > > some strange things happen (at the application level) with log files over > > > this size, presumably due to the use of a signed long. Your mention of > > > deleting previous days made me wonder. > > > > > > Cheers. > > > > > > > The size of the log file is not the error in this case. i made a script > > which runs (since a long time) daily all of the virtual servers. the > > logfiles then are compressed and archived. so it is always only one day. the > > smallest of this daily logs where it happens has about 200K, the biggest > > about 70MB. it looks like webalizer is sceewed up with the .current file. > > > > Karl > > > > > > > > > > > > -- > > > Rowan Crowe > > > camrecord.com / camdiscover.com / Sensation Internet Services > > > Melbourne, Australia > > > > > > > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-isp" in the body of the message > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-isp" in the body of the message > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 15:56: 7 2001 Delivered-To: freebsd-isp@freebsd.org Received: from metva.com.au (metva.com.au [202.0.82.1]) by hub.freebsd.org (Postfix) with ESMTP id 65BBC37B403 for ; Sun, 7 Oct 2001 15:56:03 -0700 (PDT) Received: (from enno@localhost) by metva.com.au id IAA03764; Mon, 8 Oct 2001 08:55:40 +1000 (EST) From: Enno Davids Message-Id: <200110072255.IAA03764@metva.com.au> Subject: Re: Webalizer oddness In-Reply-To: from Jim Weeks at "Oct 7, 1 04:06:32 pm" To: jim@siteplus.net (Jim Weeks) Date: Mon, 8 Oct 2001 08:55:40 +1000 (EST) Cc: freebsd-isp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL39 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org | Here is where the oddness comes in. If I copy one of the log files and | delete everything before the 5th, I can do a test run which accurately shows | all traffic for the 5th through 7th. If I delete dividual days, that | works too. If the log is run in its entirety, everything after the 4th is | ignored. It may be related to a bug I saw in webalizer at work some months back. Essentially, there was (is?) a buggy version of Opera out which emitted a malformed User-Agent string. Something like: "Opera (Linux 2.x.x (Linux 2.x.x (Linux 2.x.x (Linux 2.x.x ...." When webalizer tries to process this it crashes out as the code which copies out the User Agent string uses an unprotected copy which just looks for the ')' to terminate. By adding some code stop when the UA buffer size was reached it all worked again for me. i.e. its just another buffer overrun bug, albeit without the security concerns so many other bring with them. If you like I can post some diffs when I get to work. Alternately just look through your logs for very long log lines (which are likely the culprit). It may also be that some other fields in the log processing are similarly constructed and that webalizer may be blowing up for similar reasons elsewhere in the code. Enno. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 16: 8:58 2001 Delivered-To: freebsd-isp@freebsd.org Received: from athena.fasturl.net (athena.fasturl.net [209.32.216.150]) by hub.freebsd.org (Postfix) with ESMTP id 01ECE37B401 for ; Sun, 7 Oct 2001 16:08:55 -0700 (PDT) Received: by athena.fasturl.net (Postfix, from userid 1001) id 6ECDD66B07; Sun, 7 Oct 2001 23:09:03 +0000 (GMT) Date: Sun, 7 Oct 2001 23:09:03 +0000 From: Michael Burns To: Jim Weeks Cc: freebsd-isp@freebsd.org Subject: Re: Webalizer oddness Message-ID: <20011007230903.A95305@fasturl.net> References: <001a01c14f7f$e7587720$0a05a8c0@ooe.kmjeuro.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ; from jim@siteplus.net on Sun, Oct 07, 2001 at 06:40:13PM -0400 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Webalizer versions prior to 2.01-06 have an int rollover bug in the expression they use to compare timestamps, with the result that all records after a certain October 2001 date are ignored. I fixed the problem by patching old versions of webalizer and recompiling. You could also upgrade to a current version. A diff for 1.30 is below: ----BEGIN---- 517,518c517,518 < rec_tstamp=((jdate(rec_day,rec_month,rec_year)-epoch)*1000000)+ < (rec_hour*10000) + (rec_min*100) + rec_sec; --- > rec_tstamp=((jdate(rec_day,rec_month,rec_year)-epoch)*86400)+ > (rec_hour*3600) + (rec_min*60) + rec_sec; 4512,4513c4512,4513 < cur_tstamp=((jdate(cur_day,cur_month,cur_year)-epoch)*1000000)+ < (cur_hour*10000) + (cur_min*100) + cur_sec; --- > cur_tstamp=((jdate(cur_day,cur_month,cur_year)-epoch)*86400)+ > (cur_hour*3600) + (cur_min*60) + cur_sec; ----END------ -- Michael Burns Systems Administrator Vener Net Inc. michael@fasturl.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 18:50: 6 2001 Delivered-To: freebsd-isp@freebsd.org Received: from femail4.sdc1.sfba.home.com (femail4.sdc1.sfba.home.com [24.0.95.84]) by hub.freebsd.org (Postfix) with ESMTP id F128B37B409 for ; Sun, 7 Oct 2001 18:49:54 -0700 (PDT) Received: from veager.jwweeks.com ([65.14.122.116]) by femail4.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011008014954.PFPS2835.femail4.sdc1.sfba.home.com@veager.jwweeks.com>; Sun, 7 Oct 2001 18:49:54 -0700 Date: Sun, 7 Oct 2001 21:49:52 -0400 (EDT) From: Jim Weeks X-Sender: jim@veager.jwweeks.com To: Michael Burns Cc: freebsd-isp@freebsd.org Subject: Re: Webalizer oddness In-Reply-To: <20011007230903.A95305@fasturl.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Thanks for the diff, but I had already done the upgrade. The later version definitely fixes the problem. -- Jim Weeks On Sun, 7 Oct 2001, Michael Burns wrote: > Webalizer versions prior to 2.01-06 have an int rollover bug in the > expression they use to compare timestamps, with the result that all > records after a certain October 2001 date are ignored. I fixed the > problem by patching old versions of webalizer and recompiling. You > could also upgrade to a current version. > A diff for 1.30 is below: > > ----BEGIN---- > 517,518c517,518 > < rec_tstamp=((jdate(rec_day,rec_month,rec_year)-epoch)*1000000)+ > < (rec_hour*10000) + (rec_min*100) + rec_sec; > --- > > rec_tstamp=((jdate(rec_day,rec_month,rec_year)-epoch)*86400)+ > > (rec_hour*3600) + (rec_min*60) + rec_sec; > 4512,4513c4512,4513 > < cur_tstamp=((jdate(cur_day,cur_month,cur_year)-epoch)*1000000)+ > < (cur_hour*10000) + (cur_min*100) + cur_sec; > --- > > cur_tstamp=((jdate(cur_day,cur_month,cur_year)-epoch)*86400)+ > > (cur_hour*3600) + (cur_min*60) + cur_sec; > ----END------ > > -- > Michael Burns > Systems Administrator > Vener Net Inc. > michael@fasturl.net > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 22:56:20 2001 Delivered-To: freebsd-isp@freebsd.org Received: from workhorse.iMach.com (workhorse.iMach.com [206.127.77.89]) by hub.freebsd.org (Postfix) with ESMTP id EDE7237B403 for ; Sun, 7 Oct 2001 22:56:10 -0700 (PDT) Received: from localhost (forrestc@localhost) by workhorse.iMach.com (8.9.3/8.9.3) with ESMTP id KAA24497; Thu, 4 Oct 2001 10:56:15 -0600 (MDT) Date: Thu, 4 Oct 2001 10:56:15 -0600 (MDT) From: "Forrest W. Christian" To: "Drew J. Weaver" Cc: "'freebsd-isp@FreeBSD.ORG'" Subject: Re: firewall question In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thu, 4 Oct 2001, Drew J. Weaver wrote: > is a freebsd firewall as good as a "hardware" solution such as watchguard > fireboxes or Cisco products? Depends. Some of the "hardware" solutions are actually freebsd or similar with a gui front end taked on them. I can't actually point at any product and say they are/aren't any more secure than any others. Personally, I find most gui-configured firewalls to be scary in that they tend to be black boxes you aren't quite sure what they are doing. I highly recommend cisco IOS with FW feature set which can be added to any cisco router, with the caveat that you need to know IOS to configure the thing. That said, for most people, almost anything which runs NAT and which you don't open up any holes in is probably good enough security. Personally, I recommend a FreeBSD box with nat running and some filters to filter bogus addresses (such as ones appearing to come from you coming from the outside) at the border. About the only thing some of the commercial boxes provide that FreeBSD doesn't is in-path virus and/or java filtering and sometimes caching or monitoring of internet usage. - Forrest W. Christian (forrestc@imach.com) AC7DE ---------------------------------------------------------------------- The Innovation Machine Ltd. P.O. Box 5749 http://www.imach.com/ Helena, MT 59604 Home of PacketFlux Technogies and BackupDNS.com (406)-442-6648 ---------------------------------------------------------------------- Protect your personal freedoms - visit http://www.lp.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sun Oct 7 22:56:22 2001 Delivered-To: freebsd-isp@freebsd.org Received: from workhorse.iMach.com (workhorse.iMach.com [206.127.77.89]) by hub.freebsd.org (Postfix) with ESMTP id B83DC37B401 for ; Sun, 7 Oct 2001 22:56:15 -0700 (PDT) Received: from localhost (forrestc@localhost) by workhorse.iMach.com (8.9.3/8.9.3) with ESMTP id KAA24434; Thu, 4 Oct 2001 10:49:20 -0600 (MDT) Date: Thu, 4 Oct 2001 10:49:19 -0600 (MDT) From: "Forrest W. Christian" To: Shannon Wheeler Cc: freebsd-isp@FreeBSD.ORG Subject: Re: eggdrop In-Reply-To: <002401c14cec$da6db420$0d00a8c0@mshome.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org eggdrop is an IRC bot. Probably something one of your users started .... Depending on your policy/openness to this sort of thing you may or may not want this thing running on your machine. I've let people run them on my internal, "corporate" box where it's just my stuff, but I would be a *LOT* fussier about them on a box running publically-accessable services. On Thu, 4 Oct 2001, Shannon Wheeler wrote: > Date: Thu, 4 Oct 2001 09:54:25 -0600 > From: Shannon Wheeler > To: freebsd-isp@FreeBSD.ORG > Subject: eggdrop > > feel free to come down on me hard about this... > > yesterday my pop3 was not responding, so I telneted in and saw that > something called eggdrop1.4 was running... > > I killed it right away (shot first, ask questions later), but qpopper still > didn't respond so I rebooted. > > Eventually qpopper started responding again but it seemed to take a long > time and I had to start Apache manually. > > Was eggdrop something to do with CVS that I shouldn't have stopped? > > yes, I've looked it up. All references I've found refer to an IRC bot. - > Someone just guessed or snooped my password? > > Any suggestions for a secure telnet? > > thanks, > Shannon > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message > - Forrest W. Christian (forrestc@imach.com) AC7DE ---------------------------------------------------------------------- The Innovation Machine Ltd. P.O. Box 5749 http://www.imach.com/ Helena, MT 59604 Home of PacketFlux Technogies and BackupDNS.com (406)-442-6648 ---------------------------------------------------------------------- Protect your personal freedoms - visit http://www.lp.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Mon Oct 8 2: 5:25 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mx1.dev.itouchnet.net (devco.net [196.15.188.2]) by hub.freebsd.org (Postfix) with ESMTP id CFBAA37B406 for ; Mon, 8 Oct 2001 02:05:20 -0700 (PDT) Received: from nobody by mx1.dev.itouchnet.net with scanned_ok (Exim 3.16 #1) id 15qWMB-000JUJ-00 for freebsd-isp@freebsd.org; Mon, 08 Oct 2001 11:05:51 +0200 Received: from shell.devco.net ([196.15.188.7]) by mx1.dev.itouchnet.net with esmtp (Exim 3.16 #1) id 15qWMA-000JU5-00; Mon, 08 Oct 2001 11:05:50 +0200 Received: from bvi by shell.devco.net with local (Exim 3.20 #2) id 15qWN3-0007Ar-00; Mon, 08 Oct 2001 11:06:45 +0200 Date: Mon, 8 Oct 2001 11:06:45 +0200 From: Barry Irwin To: =?iso-8859-1?Q?Rickard_Svor=E9n?= Cc: freebsd-isp@freebsd.org Subject: Re: Dummynet & traffic-shaping Message-ID: <20011008110645.A26200@itouchlabs.com> References: <1002492561.3bc0d2916b700@webmail.xpress.se> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.2.5i In-Reply-To: <1002492561.3bc0d2916b700@webmail.xpress.se>; from rickard@xpress.se on Mon, Oct 08, 2001 at 12:09:21AM +0200 X-Checked: This message has been scanned for any virusses and unauthorized attachments. X-iScan-ID: 74911-1002531951-02168@mx1.dev.itouchnet.net version $Name: $ Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon 2001-10-08 (00:09), Rickard Svorén wrote: > Hardware = Dual PPro 200Mhz, 128 RAM, 1 Intel EtherExpress NIC & 1 Realtek 8139 > > The catv customers should have 3.5 mbps and the rest should have 500kbps. > Have tested dummynet with my workstation and pico bsd, it worked well. > But what considerations should be taken regarding hardware and queue sizes > when over 300 users are flowing through the bridge? What should be considered > as "normal". Trial and error is an option of course but this would generate > a lot of angry phonecalls to me ;-) I would suggest dumping the Realtek, and getting an additional Ether Express 10/100 (fxp) or 3Com 10/100 (xl) card. Doing this should reduce the likely hood of loosing packets and the like. Realtek cards are nice for cheap and nasty solutions, but are nit really the best for high traffic loads. Barry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Mon Oct 8 7: 5:42 2001 Delivered-To: freebsd-isp@freebsd.org Received: from anjin.xpress.se (anjin.xpress.se [195.162.64.2]) by hub.freebsd.org (Postfix) with ESMTP id E8CC137B406 for ; Mon, 8 Oct 2001 07:05:35 -0700 (PDT) Received: from rickard.xpress.se (ric.xpress.se [195.162.73.3]) by anjin.xpress.se (8.11.0/8.11.1) with ESMTP id f98E27005339; Mon, 8 Oct 2001 16:02:07 +0200 Message-Id: <4.3.2.7.2.20011008155730.0270dd18@mail.xpress.se> X-Sender: rickard@mail.xpress.se X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 08 Oct 2001 15:58:01 +0200 To: Michael Burns From: Rickard =?iso-8859-1?Q?Svor=E9n?= Subject: Re: Webalizer oddness Cc: freebsd-isp@freebsd.org In-Reply-To: <20011007230903.A95305@fasturl.net> References: <001a01c14f7f$e7587720$0a05a8c0@ooe.kmjeuro.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Having same trouble here, lucky me i have saved the source code for my=20 version (webalizer-2.00-10) so i can make the changes an roll on...But i just want to make sure, the=20 changes apply only to webalizer.c ? line 609 rec_tstamp=3D((jdate(rec_day,rec_month,rec_year)-epoch)*1000000)+ 610 (rec_hour*10000) + (rec_min*100) + rec_sec; to =3D=3D=3D> 609 rec_tstamp=3D((jdate(rec_day,rec_month,rec_year)-epoch)*86400)+ 610 (rec_hour*2600) + (rec_min*100) + rec_sec; Couldn=B4t find these in that file : < cur_tstamp=3D((jdate(cur_day,cur_month,cur_year)-epoch)*1000000)+ < (cur_hour*10000) + (cur_min*100) + cur_sec; --- > cur_tstamp=3D((jdate(cur_day,cur_month,cur_year)-epoch)*86400)+ > (cur_hour*3600) + (cur_min*60) + cur_sec; --Best Regards, Rickard Sorry for duplicates At 23:09 2001-10-07 +0000, you wrote: Webalizer versions prior to 2.01-06 have an int rollover bug in the expression they use to compare timestamps, with the result that all records after a certain October 2001 date are ignored. I fixed the problem by patching old versions of webalizer and recompiling. You could also upgrade to a current version. A diff for 1.30 is below: ----BEGIN---- 517,518c517,518 < rec_tstamp=3D((jdate(rec_day,rec_month,rec_year)-epoch)*1000000)+ < (rec_hour*10000) + (rec_min*100) + rec_sec; --- > rec_tstamp=3D((jdate(rec_day,rec_month,rec_year)-epoch)*86400)+ > (rec_hour*3600) + (rec_min*60) + rec_sec; 4512,4513c4512,4513 < cur_tstamp=3D((jdate(cur_day,cur_month,cur_year)-epoch)*1000000)+ < (cur_hour*10000) + (cur_min*100) + cur_sec; --- > cur_tstamp=3D((jdate(cur_day,cur_month,cur_year)-epoch)*86400)+ > (cur_hour*3600) + (cur_min*60) + cur_sec; ----END------ -- Michael Burns Systems Administrator Vener Net Inc. michael@fasturl.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message At 23:09 2001-10-07 +0000, you wrote: >Webalizer versions prior to 2.01-06 have an int rollover bug in the >expression they use to compare timestamps, with the result that all >records after a certain October 2001 date are ignored. I fixed the >problem by patching old versions of webalizer and recompiling. You >could also upgrade to a current version. >A diff for 1.30 is below: > >----BEGIN---- >517,518c517,518 >< rec_tstamp=3D((jdate(rec_day,rec_month,rec_year)-epoch)*1000000)= + >< (rec_hour*10000) + (rec_min*100) + rec_sec; >--- > > rec_tstamp=3D((jdate(rec_day,rec_month,rec_year)-epoch)*86400)+ > > (rec_hour*3600) + (rec_min*60) + rec_sec; >4512,4513c4512,4513 >< cur_tstamp=3D((jdate(cur_day,cur_month,cur_year)-epoch)*1000000)+ >< (cur_hour*10000) + (cur_min*100) + cur_sec; >--- > > cur_tstamp=3D((jdate(cur_day,cur_month,cur_year)-epoch)*86400)+ > > (cur_hour*3600) + (cur_min*60) + cur_sec; >----END------ > >-- >Michael Burns >Systems Administrator >Vener Net Inc. >michael@fasturl.net > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-isp" in the body of the message -Mvh , Rickard Xpress On-Line V=E4rmland AB Rickard Svor=E9n Bromsgatan 4 653 41 Karlstad rickard.svoren@xpress.se Tel : 054-521920 Fax : 054-525867 www.xpress.se To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Mon Oct 8 9:11:53 2001 Delivered-To: freebsd-isp@freebsd.org Received: from athena.fasturl.net (athena.fasturl.net [209.32.216.150]) by hub.freebsd.org (Postfix) with ESMTP id 8836D37B40A for ; Mon, 8 Oct 2001 09:11:51 -0700 (PDT) Received: by athena.fasturl.net (Postfix, from userid 1001) id 0EDB566B07; Mon, 8 Oct 2001 16:12:00 +0000 (GMT) Date: Mon, 8 Oct 2001 16:12:00 +0000 From: Michael Burns To: =?iso-8859-1?Q?Rickard_Svor=E9n?= Cc: freebsd-isp@freebsd.org Subject: Re: Webalizer oddness Message-ID: <20011008161200.A88641@fasturl.net> References: <001a01c14f7f$e7587720$0a05a8c0@ooe.kmjeuro.com> <20011007230903.A95305@fasturl.net> <4.3.2.7.2.20011008155730.0270dd18@mail.xpress.se> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4.3.2.7.2.20011008155730.0270dd18@mail.xpress.se>; from rickard@xpress.se on Mon, Oct 08, 2001 at 03:58:01PM +0200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org ON Mon, Oct 08, 2001 at 03:58:01PM +0200, Rickard Svorén wrote: > Having same trouble here, lucky me i have saved the source code for my > version (webalizer-2.00-10) > so i can make the changes an roll on...But i just want to make sure, the > changes apply only to webalizer.c ? > line Not for webalizer 2.00+; you'll want to look at "preserve.c". HTH -- Michael Burns Systems Administrator Vener Net Inc. michael@fasturl.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Mon Oct 8 10:27:10 2001 Delivered-To: freebsd-isp@freebsd.org Received: from wow.atlasta.net (wow.atlasta.net [128.241.76.20]) by hub.freebsd.org (Postfix) with ESMTP id E3C2137B406 for ; Mon, 8 Oct 2001 10:27:05 -0700 (PDT) Received: from localhost (drais@localhost) by wow.atlasta.net (8.11.2/8.11.2) with ESMTP id f98HQgN95262; Mon, 8 Oct 2001 10:26:42 -0700 (PDT) Date: Mon, 8 Oct 2001 10:26:41 -0700 (PDT) From: David Raistrick To: "Forrest W. Christian" Cc: "Drew J. Weaver" , "'freebsd-isp@FreeBSD.ORG'" Subject: Re: firewall question In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thu, 4 Oct 2001, Forrest W. Christian wrote: > On Thu, 4 Oct 2001, Drew J. Weaver wrote: > > > is a freebsd firewall as good as a "hardware" solution such as watchguard > > fireboxes or Cisco products? > About the only thing some of the commercial boxes provide that FreeBSD > doesn't is in-path virus and/or java filtering and sometimes caching or > monitoring of internet usage. But all of this could be done..admitedly with more work involved. ...david --- david raistrick (deep in the south georgia woods) drais@atlasta.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 7: 4:33 2001 Delivered-To: freebsd-isp@freebsd.org Received: from stuff.webintl.com (ns.webintl.com [209.248.144.5]) by hub.freebsd.org (Postfix) with ESMTP id 243C637B401; Tue, 9 Oct 2001 07:04:29 -0700 (PDT) Received: from [192.168.123.20] (adsl-66-136-237-161.dsl.ltrkar.swbell.net [66.136.237.161]) by stuff.webintl.com (8.9.3/8.9.3) with ESMTP id JAA28889; Tue, 9 Oct 2001 09:04:23 -0500 Mime-Version: 1.0 X-Sender: freebsd@mail.webintl.com Message-Id: Date: Tue, 9 Oct 2001 09:04:21 -0500 To: freebsd-isp@freebsd.org, freebsd-questions@freebsd.org From: Albert Everett Subject: jails and httpd/ssl virtual hosting Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Since jails are bound to a single IP address and SSL certs work on a per-IP basis, it looks like I'd need a whole jail per SSL enabled web site. Is this right? If so, is the usual strategy to jail non-SSL web sites and to leave SSL sites in the non-jail environment? Albert To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 7: 6:25 2001 Delivered-To: freebsd-isp@freebsd.org Received: from stuff.webintl.com (ns.webintl.com [209.248.144.5]) by hub.freebsd.org (Postfix) with ESMTP id C663F37B405 for ; Tue, 9 Oct 2001 07:06:23 -0700 (PDT) Received: from [192.168.123.20] (adsl-66-136-237-161.dsl.ltrkar.swbell.net [66.136.237.161]) by stuff.webintl.com (8.9.3/8.9.3) with ESMTP id JAA28925 for ; Tue, 9 Oct 2001 09:06:22 -0500 Mime-Version: 1.0 X-Sender: freebsd@mail.webintl.com Message-Id: Date: Tue, 9 Oct 2001 09:06:20 -0500 To: freebsd-isp@freebsd.org From: Albert Everett Subject: ssl web sites and jail Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Since jails are bound to a single IP address and SSL certs work on a per-IP basis, it looks like I'd need a whole jail per SSL enabled web site. Is this right? If so, is the usual strategy to jail non-SSL web sites and to leave SSL sites in the non-jail environment? Albert To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 8:59:12 2001 Delivered-To: freebsd-isp@freebsd.org Received: from nexusinternetsolutions.net (nx1.nexusinternetsolutions.net [204.50.158.10]) by hub.freebsd.org (Postfix) with SMTP id 2A63C37B407 for ; Tue, 9 Oct 2001 08:59:10 -0700 (PDT) Received: (qmail 6181 invoked from network); 9 Oct 2001 15:54:07 -0000 Received: from unknown (HELO WS1) (204.50.158.15) by nx1.nexusinternetsolutions.net with SMTP; 9 Oct 2001 15:54:07 -0000 From: "Dave" To: Subject: Compaq ProLiant DL360 experience/compatibility Date: Tue, 9 Oct 2001 11:58:42 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Compaq ProLiant DL360 Dual PIII 800mhz 1GB RAM, dual 18.2 drives (not sure what SCSI driver Compaq uses) Looking for first hand experience and any compatibility issues with 4.3 branch of FreeBSD, assume mirrored configuration of hard drives. thanks Dave To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 9: 4:57 2001 Delivered-To: freebsd-isp@freebsd.org Received: from EXCHANGE.bwalk.com (exchange.bwalk.com [139.142.15.70]) by hub.freebsd.org (Postfix) with ESMTP id 80E0D37B407 for ; Tue, 9 Oct 2001 09:04:55 -0700 (PDT) Received: by EXCHANGE.bwalk.com with Internet Mail Service (5.5.2653.19) id <4S1QKRX3>; Tue, 9 Oct 2001 10:03:46 -0600 Message-ID: <493DE418616E9D48A5DB8E9FAAE1A8CF03F5C50D@EXCHANGE.bwalk.com> From: Adam Serediuk To: 'Dave' , freebsd-isp@FreeBSD.ORG Subject: RE: Compaq ProLiant DL360 experience/compatibility Date: Tue, 9 Oct 2001 10:03:45 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Have four of the systems setup here, work great. Using 4.3-STABLE as of Jun 18. Only issue is that FreeBSD does not support the add-on smartraid 5200 controller (I think that's the model#) unless that has changed recently... The built in SCSI controller works fine however. -----Original Message----- From: Dave [mailto:dave@nexusinternetsolutions.net] Sent: Tuesday, October 09, 2001 9:59 AM To: freebsd-isp@FreeBSD.ORG Subject: Compaq ProLiant DL360 experience/compatibility Compaq ProLiant DL360 Dual PIII 800mhz 1GB RAM, dual 18.2 drives (not sure what SCSI driver Compaq uses) Looking for first hand experience and any compatibility issues with 4.3 branch of FreeBSD, assume mirrored configuration of hard drives. thanks Dave To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 10:18:31 2001 Delivered-To: freebsd-isp@freebsd.org Received: from altoonanet.altoonanet.com (ns1.altoonanet.com [12.151.19.2]) by hub.freebsd.org (Postfix) with SMTP id 115E737B403 for ; Tue, 9 Oct 2001 10:18:27 -0700 (PDT) Received: (qmail 30807 invoked from network); 10 Oct 2001 05:19:36 -0000 Received: from cr1048452-a.etob1.on.wave.home.com (HELO cr1048452a) (24.156.44.78) by freedomhosting.com with SMTP; 10 Oct 2001 05:19:36 -0000 Message-ID: <053e01c150e7$ad417460$4e2c9c18@etob1.on.wave.home.com> From: "Kevin Turner" To: Subject: Strange SMP Problem. Date: Tue, 9 Oct 2001 13:27:31 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hello, Just recently I noticed one of my webservers was not showing two CPU's when doing a top. I checked dmesg, and noticed the following: FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 1, version: 0x00170011, at 0xfec00000 cpu1 is not showing up. The machine is running 4.2-STABLE, and has been working perfectly for quite some time. I haven't made any kernel changes or any other changes I can think of. Any ideas? Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 11:36:39 2001 Delivered-To: freebsd-isp@freebsd.org Received: from otto.oss.uswest.net (otto.oss.uswest.net [204.147.85.81]) by hub.freebsd.org (Postfix) with SMTP id EC5BC37B409 for ; Tue, 9 Oct 2001 11:36:34 -0700 (PDT) Received: (qmail 24895 invoked by uid 1001); 9 Oct 2001 18:35:56 -0000 Date: Tue, 9 Oct 2001 13:35:56 -0500 From: Pete McKenna To: Kevin Turner Cc: freebsd-isp@FreeBSD.ORG Subject: Re: Strange SMP Problem. Message-ID: <20011009133555.A23419@otto.oss.qwest.net> References: <053e01c150e7$ad417460$4e2c9c18@etob1.on.wave.home.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <053e01c150e7$ad417460$4e2c9c18@etob1.on.wave.home.com>; from kevin@freedomhosting.com on Tue, Oct 09, 2001 at 01:27:31PM -0400 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org You don't say what MB you have but we had a problem with some Intel boards which were fixed by telling the MB bios to rescan for CPU's. The board apparently cached the results of a earlier CPU scan when it was running with a single CPU. Don't know if this applies to your situation but it got us running on both CPU's. Pete On Tue, Oct 09, 2001 at 01:27:31PM -0400, Kevin Turner wrote: > Hello, > > Just recently I noticed one of my webservers was not showing two CPU's when > doing a top. > > I checked dmesg, and noticed the following: > > FreeBSD/SMP: Multiprocessor motherboard > cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 > io0 (APIC): apic id: 1, version: 0x00170011, at 0xfec00000 > > cpu1 is not showing up. > > The machine is running 4.2-STABLE, and has been working perfectly for quite > some time. > I haven't made any kernel changes or any other changes I can think of. > > Any ideas? > > Kevin > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message -- Peter McKenna Qwest Internet Solutions pmckenna@qwest.net Main 612-664-4000 FAX 612-664-4770 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 11:38:57 2001 Delivered-To: freebsd-isp@freebsd.org Received: from EXCHANGE.bwalk.com (exchange.bwalk.com [139.142.15.70]) by hub.freebsd.org (Postfix) with ESMTP id 101DA37B40C for ; Tue, 9 Oct 2001 11:38:55 -0700 (PDT) Received: by EXCHANGE.bwalk.com with Internet Mail Service (5.5.2653.19) id <4S1QKS66>; Tue, 9 Oct 2001 12:37:45 -0600 Message-ID: <493DE418616E9D48A5DB8E9FAAE1A8CF03F5C510@EXCHANGE.bwalk.com> From: Adam Serediuk To: 'Kevin Turner' , freebsd-isp@freebsd.org Subject: RE: Strange SMP Problem. Date: Tue, 9 Oct 2001 12:37:44 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org If it's an ASUS board you can try going into the BIOS and changing the APIC MPS mode(Think that's it) there's either MPS 1.0 or MPS 1.4 I think(going off memory here.) I had a problem like this once with an Asus P2B-D and changing those modes seemed to help. My understanding of the MPS versions is just that MPS 1.4 has a stricter set of rules than 1.0. Again something I know nothing about. -----Original Message----- From: Kevin Turner [mailto:kevin@freedomhosting.com] Sent: Tuesday, October 09, 2001 11:28 AM To: freebsd-isp@freebsd.org Subject: Strange SMP Problem. Hello, Just recently I noticed one of my webservers was not showing two CPU's when doing a top. I checked dmesg, and noticed the following: FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 1, version: 0x00170011, at 0xfec00000 cpu1 is not showing up. The machine is running 4.2-STABLE, and has been working perfectly for quite some time. I haven't made any kernel changes or any other changes I can think of. Any ideas? Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 13: 6:38 2001 Delivered-To: freebsd-isp@freebsd.org Received: from altoonanet.altoonanet.com (ns1.altoonanet.com [12.151.19.2]) by hub.freebsd.org (Postfix) with SMTP id C15BC37B40D for ; Tue, 9 Oct 2001 13:06:33 -0700 (PDT) Received: (qmail 32791 invoked from network); 10 Oct 2001 08:07:44 -0000 Received: from cr1048452-a.etob1.on.wave.home.com (HELO cr1048452a) (24.156.44.78) by freedomhosting.com with SMTP; 10 Oct 2001 08:07:44 -0000 Message-ID: <004201c150ff$1e204b40$4e2c9c18@etob1.on.wave.home.com> From: "Kevin Turner" To: "Pete McKenna" Cc: References: <053e01c150e7$ad417460$4e2c9c18@etob1.on.wave.home.com> <20011009133555.A23419@otto.oss.qwest.net> Subject: Re: Strange SMP Problem. Date: Tue, 9 Oct 2001 16:15:18 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hello, It worked like a charm. (440GX MB) Thanks Pete! Cheers, Kevin ----- Original Message ----- From: "Pete McKenna" To: "Kevin Turner" Cc: Sent: Tuesday, October 09, 2001 2:35 PM Subject: Re: Strange SMP Problem. > You don't say what MB you have but we had a problem with some > Intel boards which were fixed by telling the MB bios to rescan > for CPU's. The board apparently cached the results of a earlier > CPU scan when it was running with a single CPU. Don't know > if this applies to your situation but it got us running on both > CPU's. > > Pete > > On Tue, Oct 09, 2001 at 01:27:31PM -0400, Kevin Turner wrote: > > Hello, > > > > Just recently I noticed one of my webservers was not showing two CPU's when > > doing a top. > > > > I checked dmesg, and noticed the following: > > > > FreeBSD/SMP: Multiprocessor motherboard > > cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 > > io0 (APIC): apic id: 1, version: 0x00170011, at 0xfec00000 > > > > cpu1 is not showing up. > > > > The machine is running 4.2-STABLE, and has been working perfectly for quite > > some time. > > I haven't made any kernel changes or any other changes I can think of. > > > > Any ideas? > > > > Kevin > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-isp" in the body of the message > > -- > Peter McKenna Qwest Internet Solutions > pmckenna@qwest.net Main 612-664-4000 > FAX 612-664-4770 > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Tue Oct 9 15:59:22 2001 Delivered-To: freebsd-isp@freebsd.org Received: from ns.overalt.no (ns.overalt.no [194.234.214.150]) by hub.freebsd.org (Postfix) with ESMTP id B468A37B409 for ; Tue, 9 Oct 2001 15:59:17 -0700 (PDT) Received: from overalt.no (goppus.overalt.no [192.168.200.25]) by ns.overalt.no (Postfix) with ESMTP id 7E3ECF9; Wed, 10 Oct 2001 00:59:59 +0200 (CEST) Message-ID: <3BC38139.8070201@overalt.no> Date: Wed, 10 Oct 2001 00:59:05 +0200 From: Marius Sorteberg User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.4) Gecko/20010919 X-Accept-Language: en-us MIME-Version: 1.0 To: Dave Cc: freebsd-isp@FreeBSD.ORG Subject: Re: Compaq ProLiant DL360 experience/compatibility References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Dave wrote: >Compaq ProLiant DL360 >Dual PIII 800mhz >1GB RAM, dual 18.2 drives (not sure what SCSI driver Compaq uses) > >Looking for first hand experience and any compatibility issues with 4.3 branch >of FreeBSD, assume mirrored configuration of hard drives. > >thanks > >Dave > I have some older Dual ProLiant boxes, and had to set OS to Linux in SmartStart (UNIX | Linux in the menu) to get SMP. No experience with DL360 thought .... Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Wed Oct 10 2:14:25 2001 Delivered-To: freebsd-isp@freebsd.org Received: from xor.aubonne.virtua.ch (virtua.sefanet.ch [195.202.225.253]) by hub.freebsd.org (Postfix) with ESMTP id E42DB37B408; Wed, 10 Oct 2001 02:14:14 -0700 (PDT) Received: from RATAMIAOU ([192.168.1.141]) by xor.aubonne.virtua.ch (8.9.3/8.8.7) with SMTP id LAA01304; Wed, 10 Oct 2001 11:14:17 +0200 Message-ID: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> From: "Marcel Prisi" To: Cc: Subject: PostgreSQL & shared memory Date: Wed, 10 Oct 2001 11:13:41 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi all ! We just published a new website running on FreeBSD 4.4 prerelease 10 august / apache 1.3.20 / php 4.0.6 / PostgreSQL 7.1.2 . In fact we moved the site from a Win2k/iis/sqlserver to BSD, and we have a HUGE amount of hits, and the machine is already getting overloaded. PostgreSQL seems like a ressource hog. The machine is a bi-PIII 866Mhz, 380Mb RAM, Adaptec 2100s. I had to stop using both keepalive in apache and persistent connections in php in order to keep the machine from swapping and not responding anymore. The load is extremely high as some SQL queries are really hard ... The first thing we'll do is install some more memory, and then a separate machine for PostgreSQL. My question is : what are the settings I should add to be able to handle about 400-500 connections to PostgreSQL ? What about shared memory & FreeBSD kernel ? Thanks for your help. Here are some parts of the current kernel configuration file : machine i386 cpu I686_CPU ident FBSD1 maxusers 128 options NMBCLUSTERS=16384 # System V shared memory and tunable parameters options SYSVSHM # include support for shared memory options SHMMAXPGS=4096 # max amount of shared memory pages (4k on i386) options SHMSEG=256 # max shared memory segments per process # System V semaphores and tunable parameters options SYSVSEM # include support for semaphores options SEMMAP=256 # amount of entries in semaphore map options SEMMNI=256 # number of semaphore identifiers in the system options SEMMNS=512 # number of semaphores in the system options SEMMNU=256 # number of undo structures in the system # System V message queues and tunable parameters options SYSVMSG # include support for message queues # To make an SMP kernel, the next two are needed options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Wed Oct 10 2:26:10 2001 Delivered-To: freebsd-isp@freebsd.org Received: from ns1.crl.go.jp (ns1.crl.go.jp [133.243.3.1]) by hub.freebsd.org (Postfix) with ESMTP id 51E3937B403; Wed, 10 Oct 2001 02:26:01 -0700 (PDT) Received: from crlgw1.crl.go.jp ([133.243.18.250]) by ns1.crl.go.jp (8.11.6/3.7W) with ESMTP id f9A9PxU26529; Wed, 10 Oct 2001 18:25:59 +0900 (JST) Received: from cain.gsoft.com.au (localhost [127.0.0.1]) by crlgw1.crl.go.jp (8.11.6/8.11.6) with ESMTP id f9A9Pwl09821; Wed, 10 Oct 2001 18:25:58 +0900 (JST) Message-ID: X-Mailer: XFMail 1.5.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> Date: Thu, 11 Oct 2001 18:24:23 +0900 (JST) From: "Daniel O'Connor" To: Marcel Prisi Subject: RE: PostgreSQL & shared memory Cc: freebsd-stable@FreeBSD.ORG, freebsd-isp@FreeBSD.ORG Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 10-Oct-2001 Marcel Prisi wrote: > about 400-500 connections to PostgreSQL ? What about shared memory & FreeBSD > kernel ? You can tune the shared memory/semaphore options in PGSQL using sysctl which may help you a bit.. ie -> [chowder 18:20] ~ > sysctl -a | grep shm kern.ipc.shmmax: 33554432 kern.ipc.shmmin: 1 kern.ipc.shmmni: 192 kern.ipc.shmseg: 128 kern.ipc.shmall: 8192 kern.ipc.shm_use_phys: 0 Also, man 7 tuning :) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 1: 6:28 2001 Delivered-To: freebsd-isp@freebsd.org Received: from xor.aubonne.virtua.ch (virtua.sefanet.ch [195.202.225.253]) by hub.freebsd.org (Postfix) with ESMTP id E45C437B405 for ; Thu, 11 Oct 2001 01:06:21 -0700 (PDT) Received: from RATAMIAOU ([192.168.1.141]) by xor.aubonne.virtua.ch (8.9.3/8.8.7) with SMTP id KAA09173; Thu, 11 Oct 2001 10:06:00 +0200 Message-ID: <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> From: "Marcel Prisi" To: "Wojciech Sobczuk" Cc: "J. Goodleaf" , "BSD-ISP" References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> <20011010144555.B28041@freebsd.hbz.pl> Subject: Re: PostgreSQL & shared memory Date: Thu, 11 Oct 2001 10:05:26 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi ! Daniel's hint did not help at all as I already had tweaked those values (thanks for his help anyway). The most important ones seem to be in the kernel config, both SEMMNI and SEMMNS. With SEMMNI=256 and SEMMNS=512 (have a look in my previous email for the other values) I could get about 400 connections, which is enough. On the other hand, I have been impressed by FreeBSD as even when the load was incredibly high (I saw up to 35.4 load average !!), I could always have an ssh console and the machine stood up. This is the first FreeBSD machine we have in such a load (we were linux-oriented up to now) and I must say that I doubt linux would have handled such a load that way). What seems interested is the kind of "deadlock" in which the machine seem to have been : we had so many requests that the machine started swapping (it has 1Gb swap) and more and more zombie processes arrived. I had to restart both apache & postgresql as the machine was unable to answer any http request. It started again everytime the machine started swapping again. In order to keep the machine in the ~2.0 - 3.0 load average, I had to stop apache from using keepalive and stop php from using persistent postgresql connections. We now added 512Mb of RAM, giving a total of 896Mb, anf I could allow keepalive & pesistent connections again, the machine does not swap anymore, and it works much better, so yes, adding RAM is one of the best solutions ! We'll set-up a second machine in the next days, so we'll separate db from http server, we'll see if it helps (I am sure it will help a bit, but not that much). I now have to see with our development department in order to optimize their code, as there are way too many sql requests, and I am sure this will be a much better solutions. Thanks to all who helped ! ----- Original Message ----- From: "Wojciech Sobczuk" To: "Marcel Prisi" Sent: Wednesday, October 10, 2001 2:45 PM Subject: Re: PostgreSQL & shared memory > On Wed, Oct 10, 2001 at 11:13:41AM +0200, Marcel Prisi wrote: > > Hi all ! > > > > We just published a new website running on FreeBSD 4.4 prerelease 10 august > > / apache 1.3.20 / php 4.0.6 / PostgreSQL 7.1.2 . > > > > hey, > > did Daniel Oconnor's hint help with anything? i'm going to setup such a > computer any day now and i'd like to know if tuning the kernel will make it > work. > > last time i setup postgresql with a heavy load (it was a 6.x on RedHat) - i > pulled it down because it clogged the box.. > > thanks, > Wojtek To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 1:11:46 2001 Delivered-To: freebsd-isp@freebsd.org Received: from ns1.infowest.com (ns1.infowest.com [204.17.177.10]) by hub.freebsd.org (Postfix) with ESMTP id 6463D37B403 for ; Thu, 11 Oct 2001 01:11:44 -0700 (PDT) Received: from there (unknown [208.186.107.143]) by ns1.infowest.com (Postfix) with SMTP id 65681212DA; Thu, 11 Oct 2001 02:11:43 -0600 (MDT) Content-Type: text/plain; charset="iso-8859-1" From: Samuel J.Greear Organization: GetMegabits, Inc. To: "Marcel Prisi" , "Wojciech Sobczuk" Subject: Re: PostgreSQL & shared memory Date: Fri, 12 Oct 2001 02:10:34 -0600 X-Mailer: KMail [version 1.3] Cc: "J. Goodleaf" , "BSD-ISP" References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> <20011010144555.B28041@freebsd.hbz.pl> <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> In-Reply-To: <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <20011011081143.65681212DA@ns1.infowest.com> Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thursday 11 October 2001 02:05 am, Marcel Prisi wrote: > Hi ! > > Daniel's hint did not help at all as I already had tweaked those values > (thanks for his help anyway). The most important ones seem to be in the > kernel config, both SEMMNI and SEMMNS. With SEMMNI=256 and SEMMNS=512 (have > a look in my previous email for the other values) I could get about 400 > connections, which is enough. > PostgreSQL is configurable in that you can define how much shared memory it is to use. Try upping this, the default is rather small. Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 1:23:18 2001 Delivered-To: freebsd-isp@freebsd.org Received: from maud.qbranch.se (maud.qbranch.se [212.209.132.5]) by hub.freebsd.org (Postfix) with ESMTP id C0D0037B408 for ; Thu, 11 Oct 2001 01:23:15 -0700 (PDT) Received: from octopussy.qbranch.se (octopussy.qbranch.se [10.0.9.16]) by maud.qbranch.se (8.12.0/8.11.4) with ESMTP id f9B8NEAC015370 for ; Thu, 11 Oct 2001 10:23:14 +0200 (MET DST) X-MimeOLE: Produced By Microsoft Exchange V6.0.4712.0 Content-Class: urn:content-classes:message Subject: SV: Alarm Clock ?? related to cron job? MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Thu, 11 Oct 2001 10:23:13 +0200 Message-ID: <24D6142B7FD6304091A160450B93F2CE12D564@octopussy.qbranch.se> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Alarm Clock ?? related to cron job? Thread-Index: AcFMJG63ZTi0Irf7EdWYIABQi9aOigGCSglg From: "Mark Rowlands" To: "Dave VanAuken" Cc: Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > -----Ursprungligt meddelande----- > Fr=E5n: Dave VanAuken [mailto:dave@hawk-systems.com] > Skickat: den 3 oktober 2001 17:59 > Till: freebsd-isp@FreeBSD.ORG > =C4mne: Alarm Clock ?? related to cron job? > Occasionally have been getting a blank email (originating > from one of our Cron jobs) that just reads "Alarm Clock"=20 > and nothing else. > Have no idea what may be causing this. That rings a bell! from Freebsd-questions - this is probably a script (perl?) timing out. If that is the case and you don't want to see these messages anymore create a handler for SIGALRM. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 1:32:44 2001 Delivered-To: freebsd-isp@freebsd.org Received: from xor.aubonne.virtua.ch (virtua.sefanet.ch [195.202.225.253]) by hub.freebsd.org (Postfix) with ESMTP id 00A6437B40A for ; Thu, 11 Oct 2001 01:32:40 -0700 (PDT) Received: from RATAMIAOU ([192.168.1.141]) by xor.aubonne.virtua.ch (8.9.3/8.8.7) with SMTP id KAA09395; Thu, 11 Oct 2001 10:31:37 +0200 Message-ID: <009801c1522f$106a5870$8d01a8c0@gastroleader.com> From: "Marcel Prisi" To: "Samuel J.Greear" , "Wojciech Sobczuk" Cc: "J. Goodleaf" , "BSD-ISP" References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> <20011010144555.B28041@freebsd.hbz.pl> <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> <20011011081143.65681212DA@ns1.infowest.com> Subject: Re: PostgreSQL & shared memory Date: Thu, 11 Oct 2001 10:31:02 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi ! I did some trys at upping the shared memory allowed to PostgreSQL (anyway, if you want to up the number of connections you HAVE to up the shared memory as every connection uses LOADS of shared memory). I had to try and add some connections up to when Postgres tells me there is not enough shared memory on the system. BTW, is there a way to know how much shared memory is used/free on a FreeBSD system ? On newer linux, shared memory is mounted as another filesystem, so you can use df / mount to have some ideas :-) Thanks for your help. ----- Original Message ----- From: "Samuel J.Greear" To: "Marcel Prisi" ; "Wojciech Sobczuk" Cc: "J. Goodleaf" ; "BSD-ISP" Sent: Friday, October 12, 2001 10:10 AM Subject: Re: PostgreSQL & shared memory > On Thursday 11 October 2001 02:05 am, Marcel Prisi wrote: > > Hi ! > > > > Daniel's hint did not help at all as I already had tweaked those values > > (thanks for his help anyway). The most important ones seem to be in the > > kernel config, both SEMMNI and SEMMNS. With SEMMNI=256 and SEMMNS=512 (have > > a look in my previous email for the other values) I could get about 400 > > connections, which is enough. > > > > > > PostgreSQL is configurable in that you can define how much shared > memory it is to use. Try upping this, the default is rather small. > > Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 2:28:20 2001 Delivered-To: freebsd-isp@freebsd.org Received: from ns1.infowest.com (ns1.infowest.com [204.17.177.10]) by hub.freebsd.org (Postfix) with ESMTP id 155F037B405 for ; Thu, 11 Oct 2001 02:28:17 -0700 (PDT) Received: from there (unknown [208.186.107.143]) by ns1.infowest.com (Postfix) with SMTP id C231621026; Thu, 11 Oct 2001 03:28:15 -0600 (MDT) Content-Type: text/plain; charset="iso-8859-1" From: Samuel J.Greear Organization: GetMegabits, Inc. To: "Marcel Prisi" , "Wojciech Sobczuk" Subject: Re: PostgreSQL & shared memory Date: Fri, 12 Oct 2001 03:27:06 -0600 X-Mailer: KMail [version 1.3] Cc: "J. Goodleaf" , "BSD-ISP" References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> <20011011081143.65681212DA@ns1.infowest.com> <009801c1522f$106a5870$8d01a8c0@gastroleader.com> In-Reply-To: <009801c1522f$106a5870$8d01a8c0@gastroleader.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <20011011092815.C231621026@ns1.infowest.com> Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thursday 11 October 2001 02:31 am, Marcel Prisi wrote: > Hi ! > > I did some trys at upping the shared memory allowed to PostgreSQL (anyway, > if you want to up the number of connections you HAVE to up the shared > memory as every connection uses LOADS of shared memory). > > I had to try and add some connections up to when Postgres tells me there is > not enough shared memory on the system. > > BTW, is there a way to know how much shared memory is used/free on a > FreeBSD system ? On newer linux, shared memory is mounted as another > filesystem, so you can use df / mount to have some ideas :-) > > Thanks for your help. > http://www.postgresql.org/idocs/index.php?kernel-resources.html ^--- one of those 'must reads' (if you haven't) .. and ipcs -a will give you pretty much all you need to know about shared memory currently in use. Sam > > On Thursday 11 October 2001 02:05 am, Marcel Prisi wrote: > > > Hi ! > > > > > > Daniel's hint did not help at all as I already had tweaked those values > > > (thanks for his help anyway). The most important ones seem to be in the > > > kernel config, both SEMMNI and SEMMNS. With SEMMNI=256 and SEMMNS=512 > > (have > > > > a look in my previous email for the other values) I could get about 400 > > > connections, which is enough. > > > > > > > > PostgreSQL is configurable in that you can define how much shared > > memory it is to use. Try upping this, the default is rather small. > > > > Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 5:20:39 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mout0.freenet.de (mout0.freenet.de [194.97.50.131]) by hub.freebsd.org (Postfix) with ESMTP id A1BCB37B403 for ; Thu, 11 Oct 2001 05:20:35 -0700 (PDT) Received: from [194.97.50.136] (helo=mx3.freenet.de) by mout0.freenet.de with esmtp (Exim 3.33 #1) id 15repF-0002s1-00; Thu, 11 Oct 2001 14:20:33 +0200 Received: from b8273.pppool.de ([213.7.130.115] helo=Magelan.Leidinger.net) by mx3.freenet.de with esmtp (Exim 3.33 #3) id 15repE-0003E7-00; Thu, 11 Oct 2001 14:20:33 +0200 Received: from Leidinger.net (netchild@localhost [127.0.0.1]) by Magelan.Leidinger.net (8.11.6/8.11.6) with ESMTP id f9BBmek04003; Thu, 11 Oct 2001 13:48:41 +0200 (CEST) (envelope-from netchild@Leidinger.net) Message-Id: <200110111148.f9BBmek04003@Magelan.Leidinger.net> Date: Thu, 11 Oct 2001 13:48:39 +0200 (CEST) From: Alexander Leidinger Subject: Re: PostgreSQL & shared memory To: marcel-lists@virtua.ch Cc: sopel@freebsd.hbz.pl, john@goodleaf.net, freebsd-isp@FreeBSD.ORG In-Reply-To: <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=iso-8859-1 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 11 Okt, Marcel Prisi wrote: > On the other hand, I have been impressed by FreeBSD as even when the load > was incredibly high (I saw up to 35.4 load average !!), I could always have > an ssh console and the machine stood up. This is the first FreeBSD machine > we have in such a load (we were linux-oriented up to now) and I must say > that I doubt linux would have handled such a load that way). > > What seems interested is the kind of "deadlock" in which the machine seem to > have been : we had so many requests that the machine started swapping (it > has 1Gb swap) and more and more zombie processes arrived. I had to restart > both apache & postgresql as the machine was unable to answer any http > request. It started again everytime the machine started swapping again. In > order to keep the machine in the ~2.0 - 3.0 load average, I had to stop > apache from using keepalive and stop php from using persistent postgresql > connections. Do you have the http accept filter enabled (man accf_http)? If you didn't use SSL, have you tried the SGI apache accelerating patches (http://oss.sgi.com/, there are bug reports for an SSL enabled apache with SGI patches, so you better didn't use it on a SSL enabled server)? This didn't solves your problems with pgsql, but it may give you a little bit more headroom. Perhaps you may also want to look at the links at "http://www.leidinger.net/cgi-bin/search.pl?q=apache&num=10" for some more apache tuning tips. > We'll set-up a second machine in the next days, so we'll separate db from > http server, we'll see if it helps (I am sure it will help a bit, but not > that much). I now have to see with our development department in order to > optimize their code, as there are way too many sql requests, and I am sure > this will be a much better solutions. Perhaps using more than one db-server may help. Bye, Alexander. -- It's not a bug, it's tradition! http://www.Leidinger.net Alexander @ Leidinger.net GPG fingerprint = C518 BC70 E67F 143F BE91 3365 79E2 9C60 B006 3FE7 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 5:33: 8 2001 Delivered-To: freebsd-isp@freebsd.org Received: from ns2.wananchi.com (mail.wananchi.com [62.8.64.4]) by hub.freebsd.org (Postfix) with ESMTP id 31AE037B408 for ; Thu, 11 Oct 2001 05:32:55 -0700 (PDT) Received: from wash by ns2.wananchi.com with local (Exim 3.33 #1) id 15rf0A-0002zX-00; Thu, 11 Oct 2001 15:31:50 +0300 Date: Thu, 11 Oct 2001 15:31:50 +0300 From: Odhiambo Washington To: Mark Russell Cc: FBSD-ISP Subject: Re: Distrubuted spam lists Message-ID: <20011011153150.S32562@ns2.wananchi.com> Mail-Followup-To: Odhiambo Washington , Mark Russell , FBSD-ISP References: <20011005014006.R19042-100000@gatekeeper.viper.net.au> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Cp3Cp8fzgozWLBWL" Content-Disposition: inline In-Reply-To: <20011005014006.R19042-100000@gatekeeper.viper.net.au> User-Agent: Mutt/1.3.22.1i X-Disclaimer: Any views expressed in this message,where not explicitly attributed otherwise, are mine alone!. X-Fortune: Chicken Little was right. X-Operating-System: FreeBSD 4.4-RC i386 X-Mailer: Mutt http://www.mutt.org/ X-Designation: Systems Administrator, Wananchi Online Ltd. X-Location: Nairobi, KE, East Africa. X-Uptime: 3:29PM up 5 days, 3:32, 6 users, load averages: 0.06, 0.12, 0.14 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --Cp3Cp8fzgozWLBWL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Mark Russell [20011004 18:42]: writing on the subject= 'Distrubuted spam lists' | Due to numerous factors I'm now running numerous different flavours of | mailer daemons. I'm after a way I can distubute spam filters to all these | boxes, does anyone have any pointers on an easy way to do this. Exim has it's system filter for this and Sendmail can employ procmail as LD= A and then achive the same but the format of the files is not the same because of= diff command language in the two. What is it that you're looking at as a quick fix. Just use one MTA and conc= entrate on it. Multiple MTAs is pain, I guess. -Wash -- Odhiambo Washington Wananchi Online Ltd., wash@wananchi.com 1st Flr Loita Hse. Tel: 254 2 313985 Loita Street., Fax: 254 2 313922 PO Box 10286,00100-NAIROBI,KE My brother Bob doesn't want to be in government -- he promised Dad he'd go= =20 straight.=20 -John Fitzgerald Kennedy=20 --Cp3Cp8fzgozWLBWL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7xZE2n7LIsuxjem8RAqkSAKCkM9dMTb8LrwJWNjf7GYceXX4ayQCfQfE6 eXpyksymong9sGKre5qB1TM= =B0df -----END PGP SIGNATURE----- --Cp3Cp8fzgozWLBWL-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 5:43: 9 2001 Delivered-To: freebsd-isp@freebsd.org Received: from ina.de (mail01.ina.de [159.51.6.53]) by hub.freebsd.org (Postfix) with ESMTP id 1698C37B403 for ; Thu, 11 Oct 2001 05:42:55 -0700 (PDT) Received: from ina-de0135.ina.de (ina-de0135.ina.de [159.51.6.55]) by ina.de (8.9.3+Sun/8.9.1) with SMTP id OAA14883 for ; Thu, 11 Oct 2001 14:42:53 +0200 (MET DST) Received: FROM coi01.coi.com BY ina-de0135.ina.de ; Thu Oct 11 14:42:53 2001 +0200 Received: from coi.de (su00996.coi.com [172.22.2.201]) by coi01.coi.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 4STHDJVW; Thu, 11 Oct 2001 14:45:30 +0200 Message-ID: <3BC593CE.FCD09FCA@coi.de> Date: Thu, 11 Oct 2001 14:42:54 +0200 From: "Mitterwald, Holger" Organization: COI GmbH X-Mailer: Mozilla 4.7 [en] (X11; I; SunOS 5.6 sun4u) X-Accept-Language: de-DE, en MIME-Version: 1.0 To: Marcel Prisi Cc: freebsd-isp@FreeBSD.ORG Subject: Re: PostgreSQL & shared memory References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Marcel Prisi wrote: > > Hi all ! > > We just published a new website running on FreeBSD 4.4 prerelease 10 august > / apache 1.3.20 / php 4.0.6 / PostgreSQL 7.1.2 . > > In fact we moved the site from a Win2k/iis/sqlserver to BSD, and we have a > HUGE amount of hits, and the machine is already getting overloaded. > PostgreSQL seems like a ressource hog. The machine is a bi-PIII 866Mhz, > 380Mb RAM, Adaptec 2100s. What do you understand as a huge amount of hits? How many percent of your pages need access to the database? > I had to stop using both keepalive in apache and persistent connections in > php in order to keep the machine from swapping and not responding anymore. > The load is extremely high as some SQL queries are really hard ... Disabling persistent connections to pgsql is in my opinion a bad idea. PostgreSQL needs a lot of time for forking a new process. Persistent connections will avoid this. I was able to speed my pgsql accesses up with persistent connections of about 4 times it was without! > The first thing we'll do is install some more memory, and then a separate > machine for PostgreSQL. > > My question is : what are the settings I should add to be able to handle > about 400-500 connections to PostgreSQL ? What about shared memory & FreeBSD > kernel ? My question is: what can you do to reduce the amounts of connections to postgreSQL? First, try to eliminate as many acesses to the databeses you can do without much loss of functionality. Often it is better to create the pages once every hour than every time it is accessed. At least only the dynamic part of the pages should created every time. Saves A LOT of time! Second, run vacuumdb regularly. PostgreSQL is a bit stupid here and running vacuumdb really increases speed if you have many updates/deletes/inserts to your database. Third, sit down at your developing system (you have one, do you???) and put before every SQL statement the "explain" command and analyze the output. There you can find out where postgres does full-table-scans and you can put an index on the table. Don't forget running "vacuumdb -z" otherwise the optimizer doesn't know anything of your indices. If you have tables with many duplicate values (lets say you save temperatures of some locations and you mainly search for the values of the locations) then put an index on the location and cluster the table for it (see man cluster). Speeding up the database will give you more speed than any kernelparameter, CPU or RAM can give. Speeding up by factor 1000 is realistic on bigger databases! If you can take the risk, disable fsync for writes. Saves a lot of time if you have many updates/deletes/inserts, too! (I think it is option "-F" at postgres) Postgres can be much faster than Oracle on some queries - if it is properly tuned! Hope this helps... Best regards, Holger To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 7:50:23 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mgw1.MEIway.com (mgw1.meiway.com [212.73.210.75]) by hub.freebsd.org (Postfix) with ESMTP id A38A237B401 for ; Thu, 11 Oct 2001 07:50:20 -0700 (PDT) Received: from mail.Go2France.com (ms1.meiway.com [212.73.210.73]) by mgw1.MEIway.com (Postfix Relay Hub) with ESMTP id 90AEE16B1E for ; Thu, 11 Oct 2001 16:50:18 +0200 (CEST) Received: from IBM-HIRXKN66F0W.Go2France.com [66.64.14.18] by mail.Go2France.com with ESMTP (SMTPD32-6.06) id A4925871007E; Thu, 11 Oct 2001 17:02:42 +0200 Message-Id: <5.1.0.14.0.20011011094458.04e82840@mail.Go2France.com> X-Sender: LConrad@Go2France.com@mail.Go2France.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 11 Oct 2001 09:49:41 -0500 To: freebsd-isp@freebsd.org From: Len Conrad Subject: Re: PostgreSQL & shared memory In-Reply-To: <3BC593CE.FCD09FCA@coi.de> References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > In fact we moved the site from a Win2k/iis/sqlserver to BSD, and we have a > > HUGE amount of hits, and the machine is already getting overloaded. > > PostgreSQL seems like a ressource hog. The machine is a bi-PIII 866Mhz, > > 380Mb RAM, Adaptec 2100s. 380 Mb of RAM for a "huge hits" MS SQL machine is way under spec'ed. It's a joke. 1 gb would be comfortable for MS SQL, esp since memory is so cheap now. You should be running x SQL on a dedicated backend server, anyway, rather than on a busy web server. 380 mb for IIS+MSSQL is pretty silly. MS SQL should start with a 512 mb machine all to itself. Len http://MenAndMice.com/DNS-training http://BIND8NT.MEIway.com : ISC BIND 8.2.4 for NT4 & W2K http://IMGate.MEIway.com : Build free, hi-perf, anti-abuse mail gateways To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 16: 5:12 2001 Delivered-To: freebsd-isp@freebsd.org Received: from inet03.citec.qld.gov.au (inet03.citec.qld.gov.au [203.5.10.10]) by hub.freebsd.org (Postfix) with ESMTP id BBAE237B407 for ; Thu, 11 Oct 2001 16:05:08 -0700 (PDT) Received: by inet03.citec.qld.gov.au; id JAA05566; Fri, 12 Oct 2001 09:05:04 +1000 (EST) Received: from citecub.citec.qld.gov.au( 131.242.4.98) by inet03.citec.qld.gov.au via smap (V2.0) id xma005169; Fri, 12 Oct 01 09:04:49 +1000 Received: from guru.citec.qld.gov.au by citecub.citec.qld.gov.au (SMI-8.6/SMI-SVR4) id JAA00089; Fri, 12 Oct 2001 09:04:46 +1000 Received: from localhost (sgcccdc@localhost) by guru.citec.qld.gov.au (8.9.3/8.9.3) with ESMTP id JAA19554; Fri, 12 Oct 2001 09:04:45 +1000 (EST) (envelope-from sgcccdc@citec.qld.gov.au) X-Authentication-Warning: guru.citec.qld.gov.au: sgcccdc owned process doing -bs Date: Fri, 12 Oct 2001 09:04:44 +1000 (EST) From: Colin Campbell To: Len Conrad Cc: Subject: Re: PostgreSQL & shared memory In-Reply-To: <5.1.0.14.0.20011011094458.04e82840@mail.Go2France.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, On Thu, 11 Oct 2001, Len Conrad wrote: > > > > In fact we moved the site from a Win2k/iis/sqlserver to BSD, and we have a > > > HUGE amount of hits, and the machine is already getting overloaded. > > > PostgreSQL seems like a ressource hog. The machine is a bi-PIII 866Mhz, > > > 380Mb RAM, Adaptec 2100s. > > 380 Mb of RAM for a "huge hits" MS SQL machine is way under spec'ed. It's a > joke. 1 gb would be comfortable for MS SQL, esp since memory is so cheap now. That was my feeling. If the machine started to swap, there isn't enough memory. Given that it's dirt cheap at the moment (I can get 256MB for A$55.00) I'd be loading the box with heaps of memory. You do not want the box to swap. Colin -- Colin Campbell Unix Support/Postmaster/Hostmaster CITEC +61 7 3006 4710 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 17:43:48 2001 Delivered-To: freebsd-isp@freebsd.org Received: from MASTER.efreightech.com (w238.z064000243.lax-ca.dsl.cnc.net [64.0.243.238]) by hub.freebsd.org (Postfix) with ESMTP id BD38537B40B for ; Thu, 11 Oct 2001 17:43:43 -0700 (PDT) Received: from localhost ([64.225.124.232]) by MASTER.efreightech.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 11 Oct 2001 17:43:47 -0700 To: freebsd-isp@FreeBSD.org From: friends@openxxx.net X-Mailer: Perl+Mail::Sender 0.7.08 by Jan Krynicky Subject: Hello, your friend recommended openxxx to you Message-ID: X-OriginalArrivalTime: 12 Oct 2001 00:43:47.0968 (UTC) FILETIME=[F452D800:01C152B6] Date: 11 Oct 2001 17:43:47 -0700 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org You have been invited to check out this adult site by one of your friends who visited us. click here , our URL is: http://www.openxxx.net/ enjoy, OpenXXX TEAM 2001 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 22:48: 0 2001 Delivered-To: freebsd-isp@freebsd.org Received: from www-es.fernuni-hagen.de (ES-i2.fernuni-hagen.de [132.176.7.81]) by hub.freebsd.org (Postfix) with ESMTP id EA8B237B405 for ; Thu, 11 Oct 2001 22:47:57 -0700 (PDT) Received: from jfh00.fernuni-hagen.de (jfh00.fernuni-hagen.de [132.176.7.6]) by www-es.fernuni-hagen.de (8.11.6/8.11.4) with SMTP id f9C5lcU13871; Fri, 12 Oct 2001 07:47:38 +0200 (CEST) (envelope-from fritz.heinrichmeyer@fernuni-hagen.de) Date: Fri, 12 Oct 2001 07:47:39 +0200 From: Fritz Heinrichmeyer To: "Marcel Prisi" Cc: freebsd-isp@FreeBSD.ORG Subject: Re: PostgreSQL & shared memory Message-Id: <20011012074739.0ac0222d.fritz.heinrichmeyer@fernuni-hagen.de> In-Reply-To: <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> <20011010144555.B28041@freebsd.hbz.pl> <006b01c1522b$7f9ad750$8d01a8c0@gastroleader.com> Organization: FernUni X-Mailer: Sylpheed version 0.6.3 (GTK+ 1.2.10; i386--freebsd4.4) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org My 0,02 Euro: It made a big difference here when we started to clean up all our databases with the pgsql-Statement: "vacuum analyze;" regularly. -- Fritz Heinrichmeyer mailto:fritz.heinrichmeyer@fernuni-hagen.de FernUniversitaet Hagen, LG ES, 58084 Hagen (Germany) tel:+49 2331/987-1166 fax:987-355 http://www-es.fernuni-hagen.de/~jfh To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Thu Oct 11 23: 5:42 2001 Delivered-To: freebsd-isp@freebsd.org Received: from pitr.tuxinternet.com (pitr.tuxinternet.com [208.32.175.113]) by hub.freebsd.org (Postfix) with ESMTP id 9B5BF37B401 for ; Thu, 11 Oct 2001 23:05:37 -0700 (PDT) Received: (from hugme@localhost) by pitr.tuxinternet.com (8.11.0/8.11.0) id f9C2EH953606 for freebsd-isp@FreeBSD.ORG; Fri, 12 Oct 2001 02:14:17 GMT (envelope-from hugme) Date: Fri, 12 Oct 2001 02:14:17 +0000 From: Hug Me To: freebsd-isp@FreeBSD.ORG Subject: Re: Hello, your friend recommended openxxx to you Message-ID: <20011012021417.A53578@pitr.tuxinternet.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from friends@openxxx.net on Thu, Oct 11, 2001 at 05:43:47PM -0700 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org is openxxx anything like openBSD? secure porn maybe? or maybe it's kinky geek porn, a mystical awk command piped to yacc humor like this is what I get for trying to set up a samba server at 2:00 in the morning. On Thu, Oct 11, 2001 at 05:43:47PM -0700, friends@openxxx.net wrote: > > > You have been invited to check out this adult site > by one of your friends who visited us. > > click here , our URL is: http://www.openxxx.net/ > enjoy, > OpenXXX TEAM 2001 > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-isp" in the body of the message -- ************************************************* hugme hugme@hugme.org http://www.hugme.org http://www.atlantacon.org PGP Public key: http://www.hugme.org/mykey.pgp To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 10:42:48 2001 Delivered-To: freebsd-isp@freebsd.org Received: from blacklamb.mykitchentable.net (ekgr-dsl2-116.citlink.net [207.173.226.116]) by hub.freebsd.org (Postfix) with ESMTP id 702E537B407 for ; Fri, 12 Oct 2001 10:42:44 -0700 (PDT) Received: from bigdaddy (bigdaddy [192.168.1.3]) by blacklamb.mykitchentable.net (Postfix) with SMTP id 8F281EE623 for ; Fri, 12 Oct 2001 10:42:43 -0700 (PDT) Message-ID: <013c01c15345$46bee910$0301a8c0@bigdaddy> From: "Drew Tomlinson" To: Subject: Bad Network Config? - Mail Refused Date: Fri, 12 Oct 2001 10:42:34 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I'm at a loss as to where to look next. I'm running Postfix on my mail server and it accepts mail from everywhere (AFAIK) except my firewall. Both the mail server and firewall are running 4.4 FBSD. The firewall is using the default Sendmail. Postfix was built from ports about two weeks ago (I can't seem to locate the command to show the version). While I was building the firewall, my network was configured as such: ISP | | IP is DHCP | ADSL Modem/Router (provides DNS & NAT) | |192.168.1.1 | ----------------- | | Firewall Server 192.168.1.2 192.168.1.4 On the firewall, I setup an alias to forward all root mail to my account on the mail server. In this config, mail was passed correctly. Once the firewall was configured, I plugged in it's second network card and now my network layout is like this: ISP | | IP is DHCP | ADSL Modem/Router (still provides DNS & NAT) |192.168.10.1 | |192.168.10.2 Firewall | |192.168.1.2 | Server 192.168.1.4 But now mail is "refused" from the firewall as shown in this command: blacksheep# echo testing | sendmail -v Recipient names must be specified blacksheep# echo testing | sendmail -v drew@mykitchentable.net drew@mykitchentable.net... Connecting to blacklamb.mykitchentable.net. via esmtp And this entry in the firewall mail log: Oct 12 05:51:15 blacksheep sendmail[5877]: f9CCpEF05877: to=drew@mykitchentable.net, ctladdr=tomlinson_dr (1000/0), delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=30008, relay=blacklamb.mykitchentable.net. [207.173.226.116], dsn=4.0.0,stat=Deferred: Connection refused by blacklamb.mykitchentable.net. However, blacklamb.mykitchentable.net is reachable from the firewall: blacksheep# ping blacklamb.mykitchentable.net PING blacklamb (192.168.1.4): 56 data bytes 64 bytes from 192.168.1.4: icmp_seq=0 ttl=64 time=1.335 ms Yet another interesting piece is that there are no corresponding entries in the mail log on the mail server. Thus I wonder where the firewall is actually attempting to send my mail? What can I do next to track down this problem? I may have missed something pretty basic as all of this is new to me. So please feel free to point out that which may be obvious to most. Thanks for any assistance, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 10:58: 5 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mail.day-light.net (dle.day-light.net [64.37.72.2]) by hub.freebsd.org (Postfix) with ESMTP id 5D65337B407 for ; Fri, 12 Oct 2001 10:58:02 -0700 (PDT) Received: from w1 (118-203.bestdsl.net [216.162.118.203]) by mail.day-light.net (Postfix) with SMTP id 72F7C43E52 for ; Fri, 12 Oct 2001 12:58:01 -0500 (CDT) Reply-To: From: "John Brooks" To: Subject: RE: Bad Network Config? - Mail Refused Date: Fri, 12 Oct 2001 12:54:54 -0500 Message-ID: <000f01c15347$03cc7940$1505010a@daylight.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 In-Reply-To: <013c01c15345$46bee910$0301a8c0@bigdaddy> Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org did you try telneting from the firewall to port 25 on the server? that should show you where things are breaking down -- John Brooks Email: john@stlbsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 11: 2: 7 2001 Delivered-To: freebsd-isp@freebsd.org Received: from omega.metrics.com (omega.metrics.com [204.138.110.1]) by hub.freebsd.org (Postfix) with ESMTP id CE52037B409 for ; Fri, 12 Oct 2001 11:02:04 -0700 (PDT) Received: from syncro.metrics.com (syncro.metrics.com [204.138.110.20]) by omega.metrics.com (8.9.3/8.9.3) with ESMTP id OAA00982 for ; Fri, 12 Oct 2001 14:02:04 -0400 (EDT) Received: by syncro.metrics.com with Internet Mail Service (5.5.2653.19) id <4TAC84C2>; Fri, 12 Oct 2001 13:58:34 -0400 Message-ID: <6B3C6B6F7AA2D511A35E0080C86993435961@syncro.metrics.com> From: "Haapanen, Tom" To: "'isp@freebsd.org'" Subject: FreeBSD, FrontPage and DES Date: Fri, 12 Oct 2001 13:58:32 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Has anyone successfully installed FrontPage 2002 extensions on FreeBSD 4.4? I managed to get through the install process -- the 2002 extensions explicitly support FreeBSD, and I now get a few steps into the FrontPage-to-Apache connection. But I fail in the authentication process. I think this is because FrontPage uses DES, while FreeBSD 4.4 defaults to MD5. Or at least that's what my net.research tells me. So, fin, I used CVS to retrieve new DES crypto sources. I successfully built the libdescrypt* libraries, copied them to /usr/lib, and symlinked the libcrypt* libdescrypt* (four separate libraries). But when I try to change a password (plain old passwd command), i get an error: murcielago 125 # passwd tomh Changing local password for tomh. New password: Retype new password: passwd: cannot set password cipher: Undefined error: 0 passwd: /etc/master.passwd: unchanged murcielago 126 # I tried with a set of libdescrypt* libraries from a FreeBSD 4.1 system, and I get exactly the same error. Once I symlink libcrypt* back to the original files, all is well -- except that I have no luck with FrontPage. Can anyone point me in the right direction? Thanks ... Tom Haapanen tomh@motorsport.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 11:39:48 2001 Delivered-To: freebsd-isp@freebsd.org Received: from blacklamb.mykitchentable.net (ekgr-dsl2-116.citlink.net [207.173.226.116]) by hub.freebsd.org (Postfix) with ESMTP id C697437B408 for ; Fri, 12 Oct 2001 11:39:44 -0700 (PDT) Received: from bigdaddy (bigdaddy [192.168.1.3]) by blacklamb.mykitchentable.net (Postfix) with SMTP id 197E7EE623; Fri, 12 Oct 2001 11:39:44 -0700 (PDT) Message-ID: <018001c1534d$3d888970$0301a8c0@bigdaddy> From: "Drew Tomlinson" To: , References: <000f01c15347$03cc7940$1505010a@daylight.net> Subject: Re: Bad Network Config? - Mail Refused Date: Fri, 12 Oct 2001 11:39:35 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org ---- Original Message ----- From: "John Brooks" To: Sent: Friday, October 12, 2001 10:54 AM Subject: RE: Bad Network Config? - Mail Refused > did you try telneting from the firewall to port 25 on the server? that > should show you where things are breaking down No but now I have. :) That works fine. blacksheep# telnet blacklamb.mykitchentable.net 25 Trying 192.168.1.4... Connected to blacklamb. Escape character is '^]'. 220 blacklamb.mykitchentable.net ESMTP Postfix So I guess I have a config problem with Postfix? Any ideas where to look? Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 11:52:44 2001 Delivered-To: freebsd-isp@freebsd.org Received: from gifw.genroco.com (genroco.com [205.254.195.202]) by hub.freebsd.org (Postfix) with ESMTP id 8445337B408 for ; Fri, 12 Oct 2001 11:52:36 -0700 (PDT) Received: from gi2.genroco.com (IDENT:root@gi2.genroco.com [192.133.120.3]) by gifw.genroco.com (8.9.3/8.9.3) with ESMTP id NAA23854; Fri, 12 Oct 2001 13:52:34 -0500 Received: from scot.genroco.com (scot.genroco.com [192.133.120.125]) by gi2.genroco.com (8.9.3/8.9.3) with SMTP id NAA19206; Fri, 12 Oct 2001 13:52:33 -0500 Message-ID: <02a901c1534f$0e140780$7d7885c0@genroco.com> From: "Scot W. Hetzel" To: "Haapanen, Tom" , "'isp@freebsd.org'" References: <6B3C6B6F7AA2D511A35E0080C86993435961@syncro.metrics.com> Subject: Re: FreeBSD, FrontPage and DES Date: Fri, 12 Oct 2001 13:52:32 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org From: "Haapanen, Tom" > Has anyone successfully installed FrontPage 2002 extensions on FreeBSD 4.4? > > I managed to get through the install process -- the 2002 extensions > explicitly support FreeBSD, and I now get a few steps into the > FrontPage-to-Apache connection. But I fail in the authentication process. > > I think this is because FrontPage uses DES, while FreeBSD 4.4 defaults to > MD5. Or at least that's what my net.research tells me. > > So, fin, I used CVS to retrieve new DES crypto sources. I successfully > built the libdescrypt* libraries, copied them to /usr/lib, and symlinked the > libcrypt* libdescrypt* (four separate libraries). But when I try to change > a password (plain old passwd command), i get an error: > As you found out you can't do this in 4.4. > Can anyone point me in the right direction? > The problem is that the FreeBSD FrontPage Ext's were build for a FreeBSD 3.x system, which had libdescrypt* linked to libcrypt*. The problem is the FP Exts generate an invalid MD5 password on a system that has either libscrypt is linked to libcrypt (FreeBSD 3.x, 4.[1-3]) or when used with libcrypt (4.4+). There are 3 possible ways to solve this problem: 1. Use the BSDi FP Exts which have the descrypt routines statically linked. 2. (I haven't tested this) Change login.conf's passwd_format to des, which will cause libcrypt (4.4+) to create DES passwords: default:\ :passwd_format=md5:\ Or you could try changing the class for the user that needs to run the FP Ext's (fpsrvadm.exe, ows*.exe) to "des_users". 3. use htpasswd to change the passwords to the correct MD5 password. You will need to locate the password files from the .htaccess files. I have reported this DES/MD5 password problem to Ready-to-Run (www.rtr.com) awhile ago, but haven't heard if they had fixed the problem with the FreeBSD FP Exts. Scot To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 12:18:18 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mail.day-light.net (dle.day-light.net [64.37.72.2]) by hub.freebsd.org (Postfix) with ESMTP id C0D8F37B43E for ; Fri, 12 Oct 2001 12:18:08 -0700 (PDT) Received: from w1 (118-203.bestdsl.net [216.162.118.203]) by mail.day-light.net (Postfix) with SMTP id E5B6F43E52 for ; Fri, 12 Oct 2001 14:18:07 -0500 (CDT) Reply-To: From: "John Brooks" To: Subject: RE: Bad Network Config? - Mail Refused Date: Fri, 12 Oct 2001 14:15:00 -0500 Message-ID: <001001c15352$340ee6a0$1505010a@daylight.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 In-Reply-To: <018001c1534d$3d888970$0301a8c0@bigdaddy> Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org once you made the telnet connection to port 25, try manually sending mail helo blacksheep mail from: drew@mykitchentable.net rcpt to: valid_user_on_system data this is a test . quit you should get a 220 response at connection, a 250 reponse for most commands, a 354 response for "data", and a 221 response at termination if that works check in /var/mail/ to see if mail ended up in the mailbox did you make sure that root is aliased to a valid user? check /usr/local/etc/postfix/main.cf and make sure you've set it up properly. some things to look at: myhostname mydomain myorigin mydestination mynetworks alias_database try setting "disable_dns_lookups = yes" to eliminate it being a dns issue personally I'd run postfix on both boxes. -- John Brooks Email: john@stlbsd.org -----Original Message----- From: owner-freebsd-isp@FreeBSD.ORG [mailto:owner-freebsd-isp@FreeBSD.ORG]On Behalf Of Drew Tomlinson Sent: Friday, October 12, 2001 1:40 PM To: john@day-light.com; isp@freebsd.org Subject: Re: Bad Network Config? - Mail Refused ---- Original Message ----- From: "John Brooks" To: Sent: Friday, October 12, 2001 10:54 AM Subject: RE: Bad Network Config? - Mail Refused > did you try telneting from the firewall to port 25 on the server? that > should show you where things are breaking down No but now I have. :) That works fine. blacksheep# telnet blacklamb.mykitchentable.net 25 Trying 192.168.1.4... Connected to blacklamb. Escape character is '^]'. 220 blacklamb.mykitchentable.net ESMTP Postfix So I guess I have a config problem with Postfix? Any ideas where to look? Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 12:29: 2 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mgw1.MEIway.com (mgw1.meiway.com [212.73.210.75]) by hub.freebsd.org (Postfix) with ESMTP id CDD0837B407 for ; Fri, 12 Oct 2001 12:28:57 -0700 (PDT) Received: from mail.Go2France.com (ms1.meiway.com [212.73.210.73]) by mgw1.MEIway.com (Postfix Relay Hub) with ESMTP id 43B5D16B13 for ; Fri, 12 Oct 2001 21:28:55 +0200 (CEST) Received: from IBM-HIRXKN66F0W.Go2France.com [66.64.14.18] by mail.Go2France.com with ESMTP (SMTPD32-6.06) id A74EF9350084; Fri, 12 Oct 2001 21:41:02 +0200 Message-Id: <5.1.0.14.0.20011012142107.02cf1d28@mail.Go2France.com> X-Sender: LConrad@Go2France.com@mail.Go2France.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 12 Oct 2001 14:28:12 -0500 To: freebsd-isp@freebsd.org From: Len Conrad Subject: 4-port T1 PCI card with integrated csu/dsu Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org We need to accept handoff of 4 x T1 and want to do it with FreeBSD. Is there such a card with support in FreeBSD? If not, suggestions? I can't get SBEI to respond(yet), does anybody know if these cards are supported in FreeBSD: http://www.sbei.net/wanxladapters.htm For an external csu/dsu, we're looking at: http://www.kentrox.com/products/datasmart_max_t1qpad/index.asp ... whose EIA530 output would connect to an etinc PCISYNC card. thanks Len http://MenAndMice.com/DNS-training http://BIND8NT.MEIway.com : ISC BIND 8.2.4 for NT4 & W2K http://IMGate.MEIway.com : Build free, hi-perf, anti-abuse mail gateways To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 13:10:10 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mail.tgd.net (rand.tgd.net [64.81.67.117]) by hub.freebsd.org (Postfix) with SMTP id 9398137B409 for ; Fri, 12 Oct 2001 13:10:03 -0700 (PDT) Received: (qmail 98284 invoked by uid 1001); 12 Oct 2001 20:09:59 -0000 Date: Fri, 12 Oct 2001 13:09:59 -0700 From: Sean Chittenden To: Marcel Prisi Cc: freebsd-isp@FreeBSD.ORG Subject: Re: PostgreSQL & shared memory Message-ID: <20011012130959.C97952@rand.tgd.net> References: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <006001c1516b$db2637b0$8d01a8c0@gastroleader.com>; from "marcel-lists@virtua.ch" on Wed, Oct 10, 2001 at = 11:13:41AM X-PGP-Key: 0x1EDDFAAD X-PGP-Fingerprint: C665 A17F 9A56 286C 5CFB 1DEA 9F4F 5CEF 1EDD FAAD X-Web-Homepage: http://sean.chittenden.org/ Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > We just published a new website running on FreeBSD 4.4 prerelease 10 august > / apache 1.3.20 / php 4.0.6 / PostgreSQL 7.1.2 . > > In fact we moved the site from a Win2k/iis/sqlserver to BSD, and we have a > HUGE amount of hits, and the machine is already getting overloaded. > PostgreSQL seems like a ressource hog. The machine is a bi-PIII 866Mhz, > 380Mb RAM, Adaptec 2100s. > > I had to stop using both keepalive in apache and persistent connections in > php in order to keep the machine from swapping and not responding anymore. > The load is extremely high as some SQL queries are really hard ... > > The first thing we'll do is install some more memory, and then a separate > machine for PostgreSQL. Whew, I was going to bing you with a clue bat if you didn't suggest getting more RAM: 380MB of RAM isn't much. All this aside, what you want to do is setup a reverse proxy server. You have these HUGE Apache and postgresql processes sitting around in RAM spending most of their time doing what? Writing their data out over the wire. The best thing you can do to speed up your webapp is to increase your kernel output buffers and/or use a reverse proxy (mod_backhand, mod_proxy, or even squid). Would you like to have a large 12-20MB process sending data out over the wire, or a small 400KB process that's mostly shared memory? -sc -- Sean Chittenden To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 13:52:28 2001 Delivered-To: freebsd-isp@freebsd.org Received: from indigo.quadrant.net (indigo.quadrant.net [207.195.92.9]) by hub.freebsd.org (Postfix) with ESMTP id 8EAC037B40A for ; Fri, 12 Oct 2001 13:52:24 -0700 (PDT) Received: from git2000 (h24-71-180-125.ss.shawcable.net [24.71.180.125]) by indigo.quadrant.net (8.9.1/8.9.1) with SMTP id OAA20369; Fri, 12 Oct 2001 14:51:40 -0600 (CST) From: "Scott Gerhardt" To: "Marcel Prisi" Cc: Subject: RE: PostgreSQL & shared memory Date: Fri, 12 Oct 2001 15:02:05 -0600 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20011012130959.C97952@rand.tgd.net> Importance: Normal Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > We just published a new website running on FreeBSD 4.4 > prerelease 10 august > > / apache 1.3.20 / php 4.0.6 / PostgreSQL 7.1.2 . > > > > In fact we moved the site from a Win2k/iis/sqlserver to BSD, > and we have a > > HUGE amount of hits, and the machine is already getting overloaded. > > PostgreSQL seems like a ressource hog. The machine is a > bi-PIII 866Mhz, > > 380Mb RAM, Adaptec 2100s. > > > > I had to stop using both keepalive in apache and persistent > connections in > > php in order to keep the machine from swapping and not > responding anymore. > > The load is extremely high as some SQL queries are really hard ... > > > > The first thing we'll do is install some more memory, and > then a separate > > machine for PostgreSQL. > > Whew, I was going to bing you with a clue bat if you didn't suggest > getting more RAM: 380MB of RAM isn't much. All this aside, what you > want to do is setup a reverse proxy server. You have these HUGE Apache > and postgresql processes sitting around in RAM spending most of their > time doing what? Writing their data out over the wire. The best thing > you can do to speed up your webapp is to increase your kernel output > buffers and/or use a reverse proxy (mod_backhand, mod_proxy, or even > squid). Would you like to have a large 12-20MB process > sending data out > over the wire, or a small 400KB process that's mostly shared > memory? -sc > You didn't give details of how your system is overloaded i.e. disk IO, memory, slow response times etc. Here are a few suggestions I can think of: 1.) Yes, add much more RAM, as Sean mentioned. RAM is cheap so max it out if you can, that will make a huge difference. 2.) Make sure that your PostgreSQL tables are indexed and your queries are optimized. Poor querys and non indexed tables design can make things crawl. 3.) See about tweaking Apache's logging to minimize open file handles and disk IO. 4.) Enable soft updates if you are comforable with soft updates. _________________________________ Scott Gerhardt, P.Geo. Gerhardt Information Technologies _________________________________ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 20:14: 8 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mail.tgd.net (rand.tgd.net [64.81.67.117]) by hub.freebsd.org (Postfix) with SMTP id 0125337B40D for ; Fri, 12 Oct 2001 20:14:04 -0700 (PDT) Received: (qmail 789 invoked by uid 1001); 13 Oct 2001 03:13:58 -0000 Date: Fri, 12 Oct 2001 20:13:58 -0700 From: Sean Chittenden To: Scott Gerhardt Cc: Marcel Prisi , freebsd-isp@FreeBSD.ORG Subject: Re: PostgreSQL & shared memory Message-ID: <20011012201358.C99867@rand.tgd.net> References: <20011012130959.C97952@rand.tgd.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ; from "scott@gerhardt-it.com" on Fri, Oct 12, 2001 at = 03:02:05PM X-PGP-Key: 0x1EDDFAAD X-PGP-Fingerprint: C665 A17F 9A56 286C 5CFB 1DEA 9F4F 5CEF 1EDD FAAD X-Web-Homepage: http://sean.chittenden.org/ Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > Whew, I was going to bing you with a clue bat if you didn't suggest > > getting more RAM: 380MB of RAM isn't much. All this aside, what you > > want to do is setup a reverse proxy server. You have these HUGE Apache > > and postgresql processes sitting around in RAM spending most of their > > time doing what? Writing their data out over the wire. The best thing > > you can do to speed up your webapp is to increase your kernel output > > buffers and/or use a reverse proxy (mod_backhand, mod_proxy, or even > > squid). Would you like to have a large 12-20MB process > > sending data out > > over the wire, or a small 400KB process that's mostly shared > > memory? -sc > > You didn't give details of how your system is overloaded i.e. disk IO, > memory, slow response times etc. > Here are a few suggestions I can think of: > > 1.) Yes, add much more RAM, as Sean mentioned. RAM is cheap so max it out > if you can, that will make a huge difference. > 2.) Make sure that your PostgreSQL tables are indexed and your queries are > optimized. Poor querys and non indexed tables design can make things crawl. > 3.) See about tweaking Apache's logging to minimize open file handles and > disk IO. > 4.) Enable soft updates if you are comforable with soft updates. Ack! Okay, real quick: this isn't a PostgreSQL problem. It's a RAM management problem. Follow this through and hopefully everyone will realize that making PostgreSQL go faster isn't going to buy anyone as much performance/ram savings as a reverse proxy server. A) Apache + mod_perl + mod_php4 = ~12-25MB processes B) Apache + mod_backhand = ~500KB (300KB shared!) Now, here's the demeaningly funny point, "Now class, which web setup would you rather have sending data out to a butt-ass-slow client that's connected over a 28.8Kbps modem? Oh! And don't forget about images and that 100KB front page." Hands down, any day of the week, everyone should've gone for setup B. The time that it takes server setup A to process a request is... conservatively, .1, maybe .2 seconds (with persistent database connections enabled on the mod_perl/php processes). Now, how much time do you think a webserver spends sending data to a client? Close to 2, maybe 3 seconds if it's a small request. I'm sure almost everyone here has images, and probably bloated java script files and an equally huge front page that was designed with Front Page or Dream Weaver. Trick question: which of the two server setups mentioned above can send data to a client? Tough one, huh? BOTH! So explain to me again why you want server setup A sending data to a client? In the time that it takes one process from server A to send data to a client, you could've handled (does finger math) 20? maybe 30 extra requests if the sending of the data was handled by server setup B. When you say this box is loaded, are we talking .5M hits? That's not too much for a box of this caliper if the architecture is done right. If you're pushing 5-6M over a single system.. then you're getting a tad gutsy and I'd think about getting another box, but I doubt that's the case and you'd probably be having disk IO issues before you had CPU or RAM issues (logging and DB at the same time don't play friendly). * Soft-updates * Some kind of a network buffer (mod_backhand/mod_proxy) * Persistent database connections (Apache::DBI) * Turn off fsync for PostgreSQL At any rate, if anyone has any questions about either HOW to setup the above system, or need further explanations as to what I just suggested, write back to the list and I'll be less snappy (should have a Guinness in hand/system by that time and I'll be less grumpy about being at work this late schooling other people on this exact concept). It's easy to quickly to point fingers at PostgreSQL because it takes up a noticeable amount of RAM. A database that uses RAM is a good thing, that's caching and will result in many speed ups when it comes to data retrieval. A heavy webserver process that's >10MB that's just sending data out over the wire? That's bad bad bad bad bad. 4-5 PostgreSQL processes + 4-5 heavy Apache processes + 100-150 itty bitty light weight Apache network buffers (mod_backhand) processes = good use of available resources. -sc -- Sean Chittenden To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 20:30:27 2001 Delivered-To: freebsd-isp@freebsd.org Received: from omega.metrics.com (omega.metrics.com [204.138.110.1]) by hub.freebsd.org (Postfix) with ESMTP id 67D8437B405 for ; Fri, 12 Oct 2001 20:30:24 -0700 (PDT) Received: from syncro.metrics.com (syncro.metrics.com [204.138.110.20]) by omega.metrics.com (8.9.3/8.9.3) with ESMTP id XAA25548; Fri, 12 Oct 2001 23:28:41 -0400 (EDT) Received: by syncro.metrics.com with Internet Mail Service (5.5.2653.19) id <4TAC843A>; Fri, 12 Oct 2001 23:25:12 -0400 Message-ID: <6B3C6B6F7AA2D511A35E0080C8699343597B@syncro.metrics.com> From: "Haapanen, Tom" To: "'Sean Chittenden'" Cc: Marcel Prisi , Scott Gerhardt , freebsd-isp@FreeBSD.ORG Subject: RE: PostgreSQL & shared memory Date: Fri, 12 Oct 2001 23:25:07 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Sean Chittenden wrote: > Ack! Okay, real quick: this isn't a PostgreSQL problem. It's a RAM > management problem. Follow this through and hopefully everyone will > realize that making PostgreSQL go faster isn't going to buy anyone as > much performance/ram savings as a reverse proxy server. > > A) Apache + mod_perl + mod_php4 = ~12-25MB processes > B) Apache + mod_backhand = ~500KB (300KB shared!) > [...] > When you say this box is loaded, are we talking .5M hits? > That's not too much for a box of this caliper if the architecture is > done right. If you're pushing 5-6M over a single system.. then you're > getting a tad gutsy and I'd think about getting another box, but I doubt > that's the case and you'd probably be having disk IO issues before you > had CPU or RAM issues (logging and DB at the same time don't play > friendly). Our Apache/Apache::ASP/DBI/MySQL web server handles about 7M hits in a typical month, and some fairly heavy database traffic. We split onto two systems: a web server (everything but MySQL) and a database server (MySQL only) - Web server: Alpha 164SX/533, 320 MB, NetBSD 1.4 - Database server: P3/700, 512 MB, FreeBSD 4.3 We haven't been clever about doing network buffering, but the web server does quite nicely, in spite of the 10 MB httpd processes. Only if the database server is not reachable does it start running out of memory. On the database side, I think MySQL is lighter that PostgreSQL -- and it's threaded. I have a hard time getting it to use all the memory on the server. :-) Our CPU utilization is low -- as long as you design your database and your SQL queries right. And this can really make a huge difference. Our front page (the most complex page on the site) takes about 1.5s to process, due to more than a dozen SQL queries. But before tuning it might have taken 10-20 seconds ... Memory is cheap. And servers aren't much more expensive ... but even the best hardware will bog down if your software sucks up all the power! ----- Tom Haapanen tomh@motorsport.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 20:54: 6 2001 Delivered-To: freebsd-isp@freebsd.org Received: from mail.tgd.net (rand.tgd.net [64.81.67.117]) by hub.freebsd.org (Postfix) with SMTP id 5860537B403 for ; Fri, 12 Oct 2001 20:54:01 -0700 (PDT) Received: (qmail 976 invoked by uid 1001); 13 Oct 2001 03:53:54 -0000 Date: Fri, 12 Oct 2001 20:53:54 -0700 From: Sean Chittenden To: "Haapanen, Tom" Cc: Marcel Prisi , Scott Gerhardt , freebsd-isp@FreeBSD.ORG Subject: Re: PostgreSQL & shared memory Message-ID: <20011012205354.D99867@rand.tgd.net> References: <6B3C6B6F7AA2D511A35E0080C8699343597B@syncro.metrics.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6B3C6B6F7AA2D511A35E0080C8699343597B@syncro.metrics.com>; from "tomh@metrics.com" on Fri, Oct 12, 2001 at = 11:25:07PM X-PGP-Key: 0x1EDDFAAD X-PGP-Fingerprint: C665 A17F 9A56 286C 5CFB 1DEA 9F4F 5CEF 1EDD FAAD X-Web-Homepage: http://sean.chittenden.org/ Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > A) Apache + mod_perl + mod_php4 = ~12-25MB processes > > B) Apache + mod_backhand = ~500KB (300KB shared!) > > [...] > > When you say this box is loaded, are we talking .5M hits? > > That's not too much for a box of this caliper if the architecture is > > done right. If you're pushing 5-6M over a single system.. then you're > > getting a tad gutsy and I'd think about getting another box, but I doubt > > that's the case and you'd probably be having disk IO issues before you > > had CPU or RAM issues (logging and DB at the same time don't play > > friendly). > > Our Apache/Apache::ASP/DBI/MySQL web server handles about 7M hits in a > typical month, and some fairly heavy database traffic. We split onto two > systems: a web server (everything but MySQL) and a database server (MySQL > only) Oooh!!! Yeah, see... I meant hits per day. Few orders of magnitude different, but, the reasoning is still the same. > - Web server: Alpha 164SX/533, 320 MB, NetBSD 1.4 > - Database server: P3/700, 512 MB, FreeBSD 4.3 Few points (if you don't mind the criticism): 1) Alpha on NetBSD? Why not FreeBSD? It should be faster, but you may have some hardware needs that require it. 2) Splitting up the work between servers based on the application, while nice, doesn't really utilize your resources correctly. If you can cluster your webservers together at all, then that'd be the best thing to do. I use mod_backhand to do this and I stick my PPro 200 right next to a few dual K6's and PIII's and the cluster takes advantage of the hardware when appropriate. http://www.modbackhand.org/ Example: http://www.modbackhand.org/backhand/ > We haven't been clever about doing network buffering, but the web server > does quite nicely, in spite of the 10 MB httpd processes. Only if the > database server is not reachable does it start running out of memory. Just think, you could increase your throughput by at least 20X just by adding such a buffer... > On the database side, I think MySQL is lighter that PostgreSQL -- and it's > threaded. I have a hard time getting it to use all the memory on the > server. :-) That's a religious debate in the making. :) It really depends on your data and your application. If you're working with less than 3 tables in any application and with data sets less than 500K, then MySQL is faster... anything beyond that, however and you're into PostgreSQL land (which isn't that much slower actually than MySQL: we're about to port everything off of MySQL to Postgres because the speed difference is less than 5% now). > Our CPU utilization is low -- as long as you design your database and your > SQL queries right. And this can really make a huge difference. Our front > page (the most complex page on the site) takes about 1.5s to process, due to > more than a dozen SQL queries. But before tuning it might have taken 10-20 > seconds ... Good point. If any of the data can be prejoined into a flat SQL table, then you could get those load times down even further (at the cost of a little disk space). > Memory is cheap. And servers aren't much more expensive ... but even the > best hardware will bog down if your software sucks up all the power! ;) -sc -- Sean Chittenden To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 21: 6:22 2001 Delivered-To: freebsd-isp@freebsd.org Received: from omega.metrics.com (omega.metrics.com [204.138.110.1]) by hub.freebsd.org (Postfix) with ESMTP id 0713037B40B for ; Fri, 12 Oct 2001 21:06:17 -0700 (PDT) Received: from syncro.metrics.com (syncro.metrics.com [204.138.110.20]) by omega.metrics.com (8.9.3/8.9.3) with ESMTP id AAA27251; Sat, 13 Oct 2001 00:05:03 -0400 (EDT) Received: by syncro.metrics.com with Internet Mail Service (5.5.2653.19) id <4TAC843Y>; Sat, 13 Oct 2001 00:01:34 -0400 Message-ID: <6B3C6B6F7AA2D511A35E0080C86993435982@syncro.metrics.com> From: "Haapanen, Tom" To: "'Sean Chittenden'" Cc: Marcel Prisi , Scott Gerhardt , freebsd-isp@FreeBSD.ORG Subject: RE: PostgreSQL & shared memory Date: Sat, 13 Oct 2001 00:01:26 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org >> - Web server: Alpha 164SX/533, 320 MB, NetBSD 1.4 >> - Database server: P3/700, 512 MB, FreeBSD 4.3 > Few points (if you don't mind the criticism): > 1) Alpha on NetBSD? Why not FreeBSD? It should be faster, but you may > have some hardware needs that require it. I set this up in late 1999, when FreeBSD/Alpha was still pretty raw. NetBSD has been extremely stable, which is my #1 objective. :-) FreeBSD/Alpha may be fine now, but since the system isn't broken ... (The FreeBSD database server came later, when I finally came to realize that MySQL really, really didn't like NetBSD threads.) > 2) Splitting up the work between servers based on the application, while > nice, doesn't really utilize your resources correctly. If you can > cluster your webservers together at all, then that'd be the best thing > to do. I use mod_backhand to do this and I stick my PPro 200 right next > to a few dual K6's and PIII's and the cluster takes advantage of the > hardware when appropriate. Sure, yes. But dedicated functionality is clean and easy to maintain -- an important factor when I have limited time available. >> We haven't been clever about doing network buffering, but the web server >> does quite nicely, in spite of the 10 MB httpd processes. Only if the >> database server is not reachable does it start running out of memory. > Just think, you could increase your throughput by at least 20X just by > adding such a buffer... We need 20x the visitors first! :-) But, yes, it's something I will look at as traffic grows. But, on the other hand, I suspect I'll end up clustering the database server side first, as it's going to be CPU-bound fairly soon. >> On the database side, I think MySQL is lighter that PostgreSQL -- and it's >> threaded. I have a hard time getting it to use all the memory on the >> server. :-) > That's a religious debate in the making. :) It really depends on your > data and your application. If you're working with less than 3 tables in > any application and with data sets less than 500K, then MySQL is > faster... anything beyond that, however and you're into PostgreSQL land > (which isn't that much slower actually than MySQL: we're about to port > everything off of MySQL to Postgres because the speed difference is less > than 5% now). Could be. :-) Three tables in any one query? Yes, I suspect that's true of 90% of the queries. Most database tables are less than 1 MB, with the exception of our fulltext search table, which is about 300 MB. >> Our CPU utilization is low -- as long as you design your database and your >> SQL queries right. And this can really make a huge difference. Our front >> page (the most complex page on the site) takes about 1.5s to process, due to >> more than a dozen SQL queries. But before tuning it might have taken 10-20 >> seconds ... > Good point. If any of the data can be prejoined into a flat SQL table, > then you could get those load times down even further (at the cost of a > little disk space). If your application allows, yes. Our data is a little too dynamic for that ... Tom Haapanen tomh@motorsport.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Fri Oct 12 21: 8:48 2001 Delivered-To: freebsd-isp@freebsd.org Received: from virtual-voodoo.com (bdsl.66.12.217.106.gte.net [66.12.217.106]) by hub.freebsd.org (Postfix) with ESMTP id 48ABB37B403 for ; Fri, 12 Oct 2001 21:08:45 -0700 (PDT) Received: (from steve@localhost) by virtual-voodoo.com (8.11.6/8.11.5) id f9D483F21915; Fri, 12 Oct 2001 23:08:03 -0500 (EST) (envelope-from steve) Date: Fri, 12 Oct 2001 23:08:03 -0500 From: Steve Ames To: Sean Chittenden Cc: "Haapanen, Tom" , Marcel Prisi , Scott Gerhardt , freebsd-isp@FreeBSD.ORG Subject: Re: PostgreSQL & shared memory Message-ID: <20011012230803.A97582@virtual-voodoo.com> References: <6B3C6B6F7AA2D511A35E0080C8699343597B@syncro.metrics.com> <20011012205354.D99867@rand.tgd.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20011012205354.D99867@rand.tgd.net> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Fri, Oct 12, 2001 at 08:53:54PM -0700, Sean Chittenden wrote: > 2) Splitting up the work between servers based on the application, while > nice, doesn't really utilize your resources correctly. If you can > cluster your webservers together at all, then that'd be the best thing > to do. I use mod_backhand to do this and I stick my PPro 200 right next > to a few dual K6's and PIII's and the cluster takes advantage of the > hardware when appropriate. > > http://www.modbackhand.org/ > > Example: > http://www.modbackhand.org/backhand/ http://www.backhand.org/ -Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message From owner-freebsd-isp Sat Oct 13 22:35:47 2001 Delivered-To: freebsd-isp@freebsd.org Received: from femail4.sdc1.sfba.home.com (femail4.sdc1.sfba.home.com [24.0.95.84]) by hub.freebsd.org (Postfix) with ESMTP id C160137B40D for ; Sat, 13 Oct 2001 22:35:45 -0700 (PDT) Received: from veager.jwweeks.com ([65.14.122.116]) by femail4.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011014053545.YKEC8041.femail4.sdc1.sfba.home.com@veager.jwweeks.com> for ; Sat, 13 Oct 2001 22:35:45 -0700 Date: Sun, 14 Oct 2001 01:35:43 -0400 (EDT) From: Jim Weeks X-Sender: jim@veager.jwweeks.com To: freebsd-isp@freebsd.org Subject: Being Used! Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I know this has nothing to do with FreeBSD, Just wondered if any others have experienced this. I notice quite a lot of user nobody perl activity on one of my servers, and set about to find where it was coming from. I quickly discovered that one of my virtual hosting clients was running "betsie-1.5.pl". This is a script developed by the BBC to convert normal (image filled) html documents to a more simple text based page. I don't have any problem with the concept, however I also discovered that it was being used to do all of the parsing work for a group of web robots owned by "googlebot.com". Any comments would be appreciated, -- Jim Weeks To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message