From owner-freebsd-fs@FreeBSD.ORG Wed Sep 20 06:20:53 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D939616A539 for ; Wed, 20 Sep 2006 06:20:53 +0000 (UTC) (envelope-from uyhgftr@classifiedtoday.com) Received: from classifiedtoday.com (cmo116.neoplus.adsl.tpnet.pl [83.31.142.116]) by mx1.FreeBSD.org (Postfix) with SMTP id B24F643D9B for ; Wed, 20 Sep 2006 06:20:44 +0000 (GMT) (envelope-from uyhgftr@classifiedtoday.com) Received: from 83.31.142.116 by classifiedtoday.com Date: Wed, 20 Sep 2006 08:20:40 +0100 From: "trever clement" X-Sender: uyhgftr@classifiedtoday.com To: Message-Id: <1289871581.fHzdvxvOOe-24707-798978242@classifiedtoday.com> MIME-Version: 1.0 Content-Type: text/plain Subject: They're like two peas in a pod. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 06:20:54 -0000 HOT ALERT - THIS ONE IS STILL CLIMBING THE CHARTS ALERT -- BREAKING MARKET NEWS REPORT ---- WBRS.PK Company Name: WILD BRUSH ENERGY Lookup: WBRS.PK Current Price: .05 Expected: STEADILY CLIMB FOR THE TOP Breaking News: Wild Brush Acquires Additional Powder River Oil & Gas Lease Wild Brush Energy (PINKSHEETS: WBRS) announces the purchase of an additional Powder River Basin Federal Oil & Gas Lease in the State of Wyoming. The lease is located on the eastern side of the Basin in Weston County. This is the fifth lease the Company now controls in the region. Wild Brush maintains its expansion strategy through acquisition of low risk, high probability oil and gas properties in proven regions such as Powder River basin. Wild Brush will continue to concentrate on the Powder River Basin area, due to the number of lease acre ages it currently controls. The Powder River Basin claims a high probability rate of discoveries and proven production as demonstrated by such companies as Chevron and Western Gas. There are presently over 12,000 gas wells, in addition to nearly 400 new wells monthly, producing over 330 billion cubic feet annually, in an area with an estimated 32 trillion cubic feet of natural gas. About WBRS.PK Wild Brush Energy is a diversified energy company whose primary goal is to identify and develop Oil & Coalbed Methane sites within the State of Wyoming. In addition, Wild Brush Energy continues to evaluate clean air alternative energy producing technologies such as Wind Power. Wild Brush trades in the U.S. under the symbol "WBRS." WATCH THIS STOCK GO HIGHER AND HIGHER -------------------------------------------------------------------------------- does 300% increase get your attention once you look you'll want to get in ----------------------- A rose by any other name would smell as sweet. To live from hand to mouth. A snail's pace. Strong as an ox. You can't teach an old dog new tricks. The sun will shine into our yard to. Walking on cloud nine. We'll hand you out to dry. Tastes like chicken. Sweet as apple pie. To live from hand to mouth. Season of mists and mellow fruitfulness. Rise and shine. Spring forward fall back. Walking on water. Too little too late. Spring to mind. Water it down. Some like carrots others like cabbage. Useless as tits on bull. Weed out. Slow as a snail. What on earth? Sow much, reap much; sow little, reap little. A tree does not move unless there is wind. Sweet as honey. Timber! A stick in the mud. From owner-freebsd-fs@FreeBSD.ORG Wed Sep 20 06:24:57 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F6D816A5AE for ; Wed, 20 Sep 2006 06:24:57 +0000 (UTC) (envelope-from qsadrkflm@classifiedtoday.com) Received: from classifiedtoday.com (cmo116.neoplus.adsl.tpnet.pl [83.31.142.116]) by mx1.FreeBSD.org (Postfix) with SMTP id A91C743E0E for ; Wed, 20 Sep 2006 06:23:44 +0000 (GMT) (envelope-from qsadrkflm@classifiedtoday.com) Received: from 83.31.142.116 by classifiedtoday.com Date: Wed, 20 Sep 2006 08:23:42 +0100 From: "Quintilla Tiana" X-Sender: qsadrkflm@classifiedtoday.com To: Message-Id: <2472746996.YzINxBO-9794237-448903278@classifiedtoday.com> MIME-Version: 1.0 Content-Type: text/plain Cc: Subject: You can't squeeze blood out of a turnip. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 06:24:57 -0000 HOT ALERT - THIS ONE IS STILL CLIMBING THE CHARTS ALERT -- BREAKING MARKET NEWS REPORT ---- WBRS.PK Company Name: WILD BRUSH ENERGY Lookup: WBRS.PK Current Price: .05 Expected: STEADILY CLIMB FOR THE TOP Breaking News: Wild Brush Acquires Additional Powder River Oil & Gas Lease Wild Brush Energy (PINKSHEETS: WBRS) announces the purchase of an additional Powder River Basin Federal Oil & Gas Lease in the State of Wyoming. The lease is located on the eastern side of the Basin in Weston County. This is the fifth lease the Company now controls in the region. Wild Brush maintains its expansion strategy through acquisition of low risk, high probability oil and gas properties in proven regions such as Powder River basin. Wild Brush will continue to concentrate on the Powder River Basin area, due to the number of lease acre ages it currently controls. The Powder River Basin claims a high probability rate of discoveries and proven production as demonstrated by such companies as Chevron and Western Gas. There are presently over 12,000 gas wells, in addition to nearly 400 new wells monthly, producing over 330 billion cubic feet annually, in an area with an estimated 32 trillion cubic feet of natural gas. About WBRS.PK Wild Brush Energy is a diversified energy company whose primary goal is to identify and develop Oil & Coalbed Methane sites within the State of Wyoming. In addition, Wild Brush Energy continues to evaluate clean air alternative energy producing technologies such as Wind Power. Wild Brush trades in the U.S. under the symbol "WBRS." WATCH THIS STOCK GO HIGHER AND HIGHER -------------------------------------------------------------------------------- Oh you have to see it (up 300%) the results speak for themselves ----------------------- Sour as a green apple. You never miss the water till the well runs dry. Your ass is grass. Water under the bridge. Sow much, reap much; sow little, reap little. When the cows come home. That's a whole new can of worms. Shiver me timber. Sweet as apple pie. Spring rain, Fall gold. Read the tea leaves. Sweet as honey. We hung them out to dry. Which came first, the chicken or the egg. The sharper is the berry, the sweeter is the wine. Tossed around like a hot potato. Spring forward fall back. You can lead a horse to water but you can't make him drink. Spill the beans. There is always next year. Thick as a brick. Welcome to my garden. Speak softly and carry a big stick. Want my place in the sun. A rose is a rose is a rose. Spring rain, Fall gold. We hung them out to dry. You can't squeeze blood out of a turnip. Putting the cart before the horse. Spring to mind. Worry often gives a small thing a big shadow. A place in the sun. Some like carrots others like cabbage. Top of the morning. Welcome to my garden. The way to a man's heart is through his stomach. From owner-freebsd-fs@FreeBSD.ORG Wed Sep 20 09:10:57 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F57616A4F3 for ; Wed, 20 Sep 2006 09:10:57 +0000 (UTC) (envelope-from deyuxbsg@tpnet.pl) Received: from achq233.neoplus.adsl.tpnet.pl (achq233.neoplus.adsl.tpnet.pl [83.10.14.233]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1560B43D88 for ; Wed, 20 Sep 2006 09:10:38 +0000 (GMT) (envelope-from deyuxbsg@tpnet.pl) Message-ID: <000d01c6dc94$a582a170$e90e0a53@m1141eb2fc7324> From: "messages" To: fs@freebsd.org Date: Wed, 20 Sep 2006 11:10:41 -0200 MIME-Version: 1.0 Content-Type: multipart/related; type="multipart/alternative"; boundary="----=_NextPart_000_0009_01C6DCA5.690B7170" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2869 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: spelled X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 09:10:57 -0000 ------=_NextPart_000_0009_01C6DCA5.690B7170 Content-Type: text/plain; charset="windows-1250" Content-Transfer-Encoding: quoted-printable search did not match any Make sure words are spelled Try different = keywords. general fewer your on can try Answers for expert help with = search. Get the latest messages emailed Sign in Web Images VideoNew News Maps more raquo Advanced Search = Members: users: Join Alerts Create group About Searched all groups Your = search did not match any Make sure words are spelled Try different = keywords. general fewer your on can try Answers for expert help with = search. Get the latest messages emailed to Alerts. Terms of Service = Privacy Home Help Sign in Web Images VideoNew News Maps more raquo Advanced = Search Members: users: Join Alerts Create group About Searched all = groups Your search did not match any Make sure words are spelled Try = different keywords. general fewer your on can try Answers for expert = help with search. Get Maps more raquo Advanced Search Members: users: Join Alerts Create group = About Searched all groups Your search did not match any Make sure words = are spelled Try different keywords. general fewer your on can try = Answers for expert help with search. Get the latest messages emailed to = Alerts. Terms of Service raquo Advanced Search Members: users: Join Alerts Create group About = Searched all groups Your search did not match any Make sure words are = spelled Try different keywords. general fewer your on can try Answers = for expert help with search. Get the latest messages emailed to Alerts. = Terms of Service Alerts Create group About Searched all groups Your search did not match = any Make sure words are spelled Try different keywords. general fewer = your on can try Answers for expert help with search. Get the latest = messages emailed to Alerts. Terms of Service Privacy About Searched all groups Your search did not ------=_NextPart_000_0009_01C6DCA5.690B7170-- From owner-freebsd-fs@FreeBSD.ORG Wed Sep 20 12:33:21 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC10316A416 for ; Wed, 20 Sep 2006 12:33:21 +0000 (UTC) (envelope-from thruway334@aol.com) Received: from 201-1-37-79.dsl.telesp.net.br (201-1-37-79.dsl.telesp.net.br [201.1.37.79]) by mx1.FreeBSD.org (Postfix) with SMTP id 664D843D88 for ; Wed, 20 Sep 2006 12:33:13 +0000 (GMT) (envelope-from thruway334@aol.com) Received: from 32.23.35.218 by 201.1.37.79; Wed, 20 Sep 2006 11:20:37 -0100 Message-ID: From: "ŒKŒ´ éD‘¾" To: freebsd-fs@freebsd.org Date: Wed, 20 Sep 2006 18:19:37 +0600 X-Mailer: AOL 1.0 for Windows US sub 442 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-IP: 19.56.88.28 Content-Type: text/plain; Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: =?iso-8859-1?q?=82=B1=82=F1=82=CE=82=F1=82=CD?= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ŒKŒ´ éD‘¾ List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 12:33:21 -0000 gZmBmoGZgUCBmYGagZmBQIGZgZqBmYFAgZmBmoGZgUCBmYGagZmBQIGZgZqBmYFAgZmBmoGZ gUCBmYGagZmBQIGZgZqBmSANCoGagZmBQIFAgUCPl5CrjkeOj41MjZCRvZCUjGaN2pKGgUCB QIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIGZgZogDQqBmYGagUCBQIFAgUCBQCAg krSBRZdMlryCyIjAkFOBRYjAkVOBRZdEl8eDVINDg2eBXIFcgUAgIIFAgUCBQIFAgUCBmoGZ IA0KgZqBmYFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQJBejJWCyJf2iKSBYIGblOmCzJf2gUCB QCAgICAgICAgICAgICAggZmBmiANCoGZgZqBmYFAgZmBmoGZgUCBmYGagZmBQIGZgZqBmYFA gZmBmoGZgUCBmYGagZmBQIGZgZqBmYFAgZmBmoGZgUCBmYGagZkgDQoNCiAgICAgICAgICAg ICAgICAgICAgIIFAgUAgiq6RU4lpi3aWs5e/k/yM+w0KgUCBQIFAgUCBQIFAgUCBQIFAgUCB QIFAgUCBq4GrgauBq4GrgauBq4GrDQqBQIFAgUCBmpxjgZqBQGh0dHA6Ly9zeC1pLmNvbS9z ZngvaW5kZXguaHRtbD9tZWRpYT1wYzM0MyAggZqcY4GaDQoNCoGagWOBY4FjgZmBY4FjgWOB moFjgWOBY4GZgWOBY4FjgZqBY4FjgWOBmYFjgWOBY4GagWOBY4FjgZmBY4FjgWOBmg0KgYOQ Vo7Qie+QbILMgm6Ca4FBkFaPl45xkeWQtoFBg3SDioFbg16BW4FBiPqXkIrFjOyObYFBjuGJ nIKzgvGCyILHgYQNCoFAgUCErISqhK2ErISqhK2ErISqhK2ErISqhK2ErISqhK2ErISqhK2E rISqhK2ErISqhK2ErISqhK2ErISqhK0gDQqBQIFAhKuRvYSrhKuQlISrhKuTb4SrhKuYXoSr hKuShoSrhKuBSYSrhKuJ74SrhKuCooSrhKuV+oSrhKuR6ISrIA0KgUCBQISvhKqEroSvhKqE roSvhKqEroSvhKqEroSvhKqEroSvhKqEroSvhKqEroSvhKqEroSvhKqEroSvhKqEriAgICAg ICAgICAgICAgICAgICANCiAgICAgICAgICAgICAgICAgICAgICAgICAgiq6RU4lpi3aWs5e/ k/yM+w0KgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAICCBq4GrgauBq4GrgauBq4GrDQqBQIFA gUCBmpxjgZqBQGh0dHA6Ly9zeC1pLmNvbS9zZngvaW5kZXguaHRtbD9tZWRpYT1wYzM0MyAg gZqcY4GaDQoNCiKBmYFCgUU6KjqBRIFFgZosgUKBRToqOoFEgZmBQoFFOio6gUSBRYGaLIFF OoGZgUIqOoFEgUWBmiyBQoFFOio6gUSBRQ0KgZ+Bn4GfgZ+Bn4FAgUCBQIFAgUCBQIFAgUCB QIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgZ+Bn4GfgZ+Bnw0KgUCBn4GfgZ+B n4GfgUCBQIFAgUCBQIFAk2+YXo+XkKuO0oLMjr+CqojhgqSBSYFAgUCBQIFAgUCBQIGfgZ+B n4GfgZ8NCoFAgUCBn4GfgZ+Bn4GfgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCB QIFAgUCBQIFAgZ+Bn4GfgZ+Bnw0KgZqBgYGagYGBmoGBgZqBgYGagYGBmoGBgZqBgYGagYGB moGBgZqBgYGagYGBmoGBgZqBgYGagYGBmoGBgZqBgYGagYGBmg0KhKyEqoSqhK2We4tDgsWD SoOPg0ODQ5Tej5eBRYNag3SDjILwjeyC6YLIgueDUoNSgrWCqYLIgqKB9IFAhKyEqoSqhK0N CoSrgV+BXoSrgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCB QIFAgUCBQISrgV+BXoSrDQqEr4SqhKqEroFAIGh0dHA6Ly9zeC1pLmNvbS9zZngvaW5kZXgu aHRtbD9tZWRpYT1wYzM0MyAggUAggUCEr4SqhKqErg0KDQqBmoFjgWOBY4GZgWOBY4FjgZqB Y4FjgWOBmYFjgWOBY4GagWOBY4FjgZmBY4FjgWOBmoFjgWOBY4GZgWOBY4FjgZoNCg0KgUCE rISxhK2BQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQCAgICAg ICAghKyEsYStgUAgDQqBQISwhLSEsoFAgqiCy4KmjG6BQUdBTIxugUGDjYOKjG6BQYtNlfuC zYLHguqC8JFfgqSBSIFIgUggICCEsIS0hLIgDQqBQISvhLOEroFAgUCBQIFAgUCBQIFAgUCB QIFAgUCBQIFAgUCBQIFAgUCBQIFAgUCBQIFAICAgICAgICCEr4SzhK4gDQogICAgICAgICAg IGh0dHA6Ly9zeC1pLmNvbS9zZngvaW5kZXguaHRtbD9tZWRpYT1wYzM0Mw0KDQqBmoFjgWOB Y4GZgWOBY4FjgZqBY4FjgWOBmYFjgWOBY4GagWOBY4FjgZmBY4FjgWOBmoFjgWOBY4GZgWOB Y4FjgZoNCg0K From owner-freebsd-fs@FreeBSD.ORG Wed Sep 20 13:37:55 2006 Return-Path: X-Original-To: fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E1DAF16A412 for ; Wed, 20 Sep 2006 13:37:55 +0000 (UTC) (envelope-from maozujz@rima-tde.net) Received: from 157.Red-83-59-35.dynamicIP.rima-tde.net (157.Red-83-59-35.dynamicIP.rima-tde.net [83.59.35.157]) by mx1.FreeBSD.org (Postfix) with ESMTP id 066CA43D9F for ; Wed, 20 Sep 2006 13:37:28 +0000 (GMT) (envelope-from maozujz@rima-tde.net) Message-ID: <000f01c6dcb9$e90ea770$9d233b53@HURTADO> From: "" To: fs@FreeBSD.org Date: Wed, 20 Sep 2006 15:37:26 -0200 MIME-Version: 1.0 Content-Type: multipart/related; type="multipart/alternative"; boundary="----=_NextPart_000_000B_01C6DCCA.AC977770" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2869 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: (no subject) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 13:37:56 -0000 ------=_NextPart_000_000B_01C6DCCA.AC977770 Content-Type: text/plain; charset="windows-1250" Content-Transfer-Encoding: quoted-printable ------=_NextPart_000_000B_01C6DCCA.AC977770-- From owner-freebsd-fs@FreeBSD.ORG Wed Sep 20 13:47:18 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CF5E16A4E1 for ; Wed, 20 Sep 2006 13:47:18 +0000 (UTC) (envelope-from zana@altacocina.com) Received: from catv-566502a8.catv.broadband.hu (catv-566502a8.catv.broadband.hu [86.101.2.168]) by mx1.FreeBSD.org (Postfix) with SMTP id 2CAB843D9B for ; Wed, 20 Sep 2006 13:47:06 +0000 (GMT) (envelope-from zana@altacocina.com) Received: from altacocina.com (altacocina-com-bk.mr.outblaze.com [64.62.181.93]) by catv-566502a8.catv.broadband.hu (Postfix) with ESMTP id 765DF9C68F for ; Wed, 20 Sep 2006 17:57:28 +0400 Message-ID: <001101c6dcbc$3978e5eb$b6081480@altacocina.com> From: Ìàêàð Åìåëüÿíîâè÷ Êàðïóøèí To: Fs Date: Wed, 20 Sep 2006 17:57:28 +0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2720.3000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1123 X-AntiVirus: OK! AntiVir MailGate Version 2.0.1; AVE: 6.15.0.0; VDF: 6.15.0.6 X-Antivirus: avast! (VPS 0638-0, 2006.09.19), Outbound message X-Antivirus-Status: Clean Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Н а л о г и , С о б с т в е н н о с т ь , Б е з о п а с н о с т ь , О т в е т с т в е н н о т ь X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 13:47:18 -0000 =cc=e0=ea=e0=f0 =c5=ec=e5=eb=fc=ff=ed=ee=e2=e8=f7 =ca=e0=f0=ef=f3=f8=e8=ed= From owner-freebsd-fs@FreeBSD.ORG Thu Sep 21 20:00:08 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7185216A4F3 for ; Thu, 21 Sep 2006 20:00:08 +0000 (UTC) (envelope-from james@jlauser.net) Received: from pancake.jlauser.net (pancake.jlauser.net [24.105.170.110]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3831743D5E for ; Thu, 21 Sep 2006 19:59:57 +0000 (GMT) (envelope-from james@jlauser.net) Received: from [IPv6:::1] (cpe-72-226-85-167.nycap.res.rr.com [72.226.85.167]) (authenticated bits=0) by pancake.jlauser.net (8.13.6/8.13.6) with ESMTP id k8LJxttZ092502 (version=TLSv1/SSLv3 cipher=EDH-DSS-DES-CBC3-SHA bits=168 verify=NO) for ; Thu, 21 Sep 2006 15:59:56 -0400 (EDT) (envelope-from james@jlauser.net) Mime-Version: 1.0 (Apple Message framework v752.2) To: freebsd-fs@freebsd.org Message-Id: <854C78DB-2099-4DA5-9E3B-F30D6947C532@jlauser.net> Content-Type: multipart/mixed; boundary=Apple-Mail-2-172968154 From: James Lauser Date: Thu, 21 Sep 2006 15:59:54 -0400 X-Mailer: Apple Mail (2.752.2) Subject: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Sep 2006 20:00:08 -0000 --Apple-Mail-2-172968154 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello. I've been having some trouble with snapshots on my FreeBSD 6.1/amd64 system. Basically, I have this system set up with a 3ware RAID card and several disks, and use it to collect backups from my other FreeBSD server (sparc64) and three Macs via rsync.. Every night, I have a script generate a snapshot of the RAID's file system, and those snapshots are kept on the system for one week before being removed (i.e. there are always 7 snapshots present on the system), so I can recover files that were accidentally removed or changed. The problem is that when a large number of files are removed or changed on the file system, the corresponding files in the snapshot get corrupted. This, obviously, makes the snapshots quite useless. After some searching, I've found a bug report filed last year that describes this problem exactly, though the log of that report does not suggest that anything has been done with it. That report is at http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/90512 Any help would be greatly appreciated. A test I ran showing the problem, plus the output of uname -a and dmesg.boot is attached. Thanks in advance. -- James L. Lauser james@jlauser.net +--------------+ | +--------+ +-+ | | + \ | | | | | | - | | | | | | | + / | | | | +--------+ + +-+ +--------------+ --Apple-Mail-2-172968154 Content-Transfer-Encoding: 7bit Content-Type: text/plain; x-mac-type=54455854; x-unix-mode=0755; x-mac-creator=74747874; name=snap.txt Content-Disposition: attachment; filename=snap.txt Sledge# cd /raid Sledge# touch foo Sledge# ls -l foo -rw-r--r-- 1 root wheel 0 Sep 18 14:07 foo Sledge# mksnap_ffs /raid /raid/.snap/snap Sledge# rm foo Sledge# mdconfig -a -t vnode -f /raid/.snap/snap -u 4 WARNING: opening backing store: /raid/.snap/snap readonly Sledge# mount -r /dev/md4 /mnt Sledge# cd /mnt Sledge# ls -l foo ls: foo: Bad file descriptor Sledge# Sledge# Sledge# Sledge# uname -a FreeBSD Sledge.jlauser.net 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 #4: Wed Sep 6 23:30:56 EDT 2006 root@Sledge.jlauser.net:/usr/obj/usr/src/sys/GENERIC amd64 Sledge# cat /var/run/dmesg.boot Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RELEASE-p6 #4: Wed Sep 6 23:30:56 EDT 2006 root@Sledge.jlauser.net:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) 64 Processor 2800+ (1799.95-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0xfc0 Stepping = 0 Features=0x78bfbff AMD Features=0xe0500800 real memory = 536805376 (511 MB) avail memory = 506322944 (482 MB) ACPI APIC Table: ioapic0 irqs 0-23 on motherboard kbd0 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 agp0: mem 0xf0000000-0xf7ffffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci0: at device 8.0 (no driver attached) twe0: <3ware Storage Controller. Driver version 1.50.01.002> port 0xe000-0xe00f mem 0xfb000000-0xfb7fffff irq 17 at device 9.0 on pci0 twe0: [GIANT-LOCKED] twe0: 4 ports, Firmware FE7X 1.05.00.023, BIOS BE7X 1.08.00.036 atapci0: port 0xe100-0xe107,0xe200-0xe203,0xe300-0xe307,0xe400-0xe403,0xe500-0xe50f,0xe600-0xe6ff irq 20 at device 15.0 on pci0 ata2: on atapci0 ata3: on atapci0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe700-0xe70f at device 15.1 on pci0 ata0: on atapci1 ata1: on atapci1 uhci0: port 0xe800-0xe81f irq 21 at device 16.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xe900-0xe91f irq 21 at device 16.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xea00-0xea1f irq 21 at device 16.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xeb00-0xeb1f irq 21 at device 16.3 on pci0 uhci3: [GIANT-LOCKED] usb3: on uhci3 usb3: USB revision 1.0 uhub3: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xfb800000-0xfb8000ff irq 21 at device 16.4 on pci0 ehci0: [GIANT-LOCKED] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered isab0: at device 17.0 on pci0 isa0: on isab0 vr0: port 0xec00-0xecff mem 0xfb801000-0xfb8010ff irq 23 at device 18.0 on pci0 miibus0: on vr0 ukphy0: on miibus0 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:50:70:e8:3d:9f fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FAST] sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 orm0: at iomem 0xc0000-0xc7fff,0xd0000-0xd0fff,0xef000-0xeffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1799954629 Hz quality 800 Timecounters tick every 1.000 msec module_register_init: MOD_LOAD (amr_linux, 0xffffffff80620710, 0) error 6 ad4: 39205MB at ata2-master SATA150 twed0: on twe0 twed0: 343417MB (703318656 sectors) Trying to mount root from ufs:/dev/ad4s1a Sledge# --Apple-Mail-2-172968154 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed --Apple-Mail-2-172968154-- From owner-freebsd-fs@FreeBSD.ORG Thu Sep 21 20:42:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7795716A407 for ; Thu, 21 Sep 2006 20:42:58 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0A1EA43D67 for ; Thu, 21 Sep 2006 20:42:51 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k8LKgpJo096975; Thu, 21 Sep 2006 15:42:52 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4512F957.2090205@centtech.com> Date: Thu, 21 Sep 2006 15:43:03 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: James Lauser References: <854C78DB-2099-4DA5-9E3B-F30D6947C532@jlauser.net> In-Reply-To: <854C78DB-2099-4DA5-9E3B-F30D6947C532@jlauser.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1919/Thu Sep 21 10:51:22 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Sep 2006 20:42:58 -0000 On 09/21/06 14:59, James Lauser wrote: > Hello. > > I've been having some trouble with snapshots on my FreeBSD 6.1/amd64 > system. Basically, I have this system set up with a 3ware RAID card > and several disks, and use it to collect backups from my other > FreeBSD server (sparc64) and three Macs via rsync.. > > Every night, I have a script generate a snapshot of the RAID's file > system, and those snapshots are kept on the system for one week > before being removed (i.e. there are always 7 snapshots present on > the system), so I can recover files that were accidentally removed or > changed. > > The problem is that when a large number of files are removed or > changed on the file system, the corresponding files in the snapshot > get corrupted. This, obviously, makes the snapshots quite useless. > > After some searching, I've found a bug report filed last year that > describes this problem exactly, though the log of that report does > not suggest that anything has been done with it. That report is at > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/90512 > > Any help would be greatly appreciated. A test I ran showing the > problem, plus the output of uname -a and dmesg.boot is attached. > > Thanks in advance. > Sledge# cd /raid > Sledge# touch foo > Sledge# ls -l foo > -rw-r--r-- 1 root wheel 0 Sep 18 14:07 foo > Sledge# mksnap_ffs /raid /raid/.snap/snap > Sledge# rm foo > Sledge# mdconfig -a -t vnode -f /raid/.snap/snap -u 4 > WARNING: opening backing store: /raid/.snap/snap readonly > Sledge# mount -r /dev/md4 /mnt > Sledge# cd /mnt > Sledge# ls -l foo > ls: foo: Bad file descriptor > Sledge# > Sledge# > Sledge# > Sledge# uname -a > FreeBSD Sledge.jlauser.net 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 #4: Wed Sep 6 23:30:56 EDT 2006 root@Sledge.jlauser.net:/usr/obj/usr/src/sys/GENERIC amd64 Hmm.. Interesting, it seems to work fine for me: [root@neutrino /tmp]# dd if=/dev/zero of=TESTDISK bs=1m count=100 100+0 records in 100+0 records out 104857600 bytes transferred in 3.390829 secs (30923882 bytes/sec) [root@neutrino /tmp]# mdconfig -a -t vnode -f ./TESTDISK md0 [root@neutrino /tmp]# newfs -U /dev/md0 /dev/md0: 100.0MB (204800 sectors) block size 16384, fragment size 2048 using 4 cylinder groups of 25.02MB, 1601 blks, 3264 inodes. with soft updates super-block backups (for fsck -b #) at: 160, 51392, 102624, 153856 [root@neutrino /tmp]# mount /dev/md0 /mnt [root@neutrino /tmp]# touch /mnt/foo [root@neutrino /tmp]# mksnap_ffs /mnt/ /mnt/.snap/snap [root@neutrino /tmp]# mdconfig -a -t vnode -f /mnt/.snap/snap WARNING: opening backing store: /mnt/.snap/snap readonly md1 [root@neutrino /tmp]# mount -r /dev/md1 /mnt2 [root@neutrino /tmp]# cd /mnt2 [root@neutrino /mnt2]# ls -l foo -rw-r--r-- 1 root wheel 0 Sep 21 15:37 foo [root@neutrino /mnt2]# Can you unmount /raid, and do an fsck -y /raid and make sure it's clean? Also - are you using quotas? How big is the filesystem? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Thu Sep 21 22:18:51 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8A81216A403 for ; Thu, 21 Sep 2006 22:18:51 +0000 (UTC) (envelope-from james@jlauser.net) Received: from pancake.jlauser.net (pancake.jlauser.net [24.105.170.110]) by mx1.FreeBSD.org (Postfix) with ESMTP id 50C9443D6A for ; Thu, 21 Sep 2006 22:18:47 +0000 (GMT) (envelope-from james@jlauser.net) Received: from [IPv6:::1] (cpe-72-226-85-167.nycap.res.rr.com [72.226.85.167]) (authenticated bits=0) by pancake.jlauser.net (8.13.6/8.13.6) with ESMTP id k8LMIkPC093612 (version=TLSv1/SSLv3 cipher=EDH-DSS-DES-CBC3-SHA bits=168 verify=NO) for ; Thu, 21 Sep 2006 18:18:46 -0400 (EDT) (envelope-from james@jlauser.net) Mime-Version: 1.0 (Apple Message framework v752.2) In-Reply-To: <4512F957.2090205@centtech.com> References: <854C78DB-2099-4DA5-9E3B-F30D6947C532@jlauser.net> <4512F957.2090205@centtech.com> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <8ED0D713-1023-4246-BCFE-FE318A287866@jlauser.net> Content-Transfer-Encoding: 7bit From: James Lauser Date: Thu, 21 Sep 2006 18:18:45 -0400 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.752.2) Subject: Re: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Sep 2006 22:18:51 -0000 fsck -y shows no issues with /raid. Quotas are not enabled. The file system is 325 GB. -- James L. Lauser james@jlauser.net +--------------+ | +--------+ +-+ | | + \ | | | | | | - | | | | | | | + / | | | | +--------+ + +-+ +--------------+ On Sep 21, 2006, at 16:43 , Eric Anderson wrote: > On 09/21/06 14:59, James Lauser wrote: >> Hello. >> I've been having some trouble with snapshots on my FreeBSD 6.1/ >> amd64 system. Basically, I have this system set up with a 3ware >> RAID card and several disks, and use it to collect backups from >> my other FreeBSD server (sparc64) and three Macs via rsync.. >> Every night, I have a script generate a snapshot of the RAID's >> file system, and those snapshots are kept on the system for one >> week before being removed (i.e. there are always 7 snapshots >> present on the system), so I can recover files that were >> accidentally removed or changed. >> The problem is that when a large number of files are removed or >> changed on the file system, the corresponding files in the >> snapshot get corrupted. This, obviously, makes the snapshots >> quite useless. >> After some searching, I've found a bug report filed last year >> that describes this problem exactly, though the log of that >> report does not suggest that anything has been done with it. >> That report is at http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/ >> 90512 >> Any help would be greatly appreciated. A test I ran showing the >> problem, plus the output of uname -a and dmesg.boot is attached. >> Thanks in advance. > > >> Sledge# cd /raid >> Sledge# touch foo >> Sledge# ls -l foo >> -rw-r--r-- 1 root wheel 0 Sep 18 14:07 foo >> Sledge# mksnap_ffs /raid /raid/.snap/snap >> Sledge# rm foo >> Sledge# mdconfig -a -t vnode -f /raid/.snap/snap -u 4 >> WARNING: opening backing store: /raid/.snap/snap readonly >> Sledge# mount -r /dev/md4 /mnt >> Sledge# cd /mnt >> Sledge# ls -l foo >> ls: foo: Bad file descriptor >> Sledge# Sledge# Sledge# Sledge# uname -a >> FreeBSD Sledge.jlauser.net 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 >> #4: Wed Sep 6 23:30:56 EDT 2006 root@Sledge.jlauser.net:/usr/ >> obj/usr/src/sys/GENERIC amd64 > > Hmm.. Interesting, it seems to work fine for me: > > [root@neutrino /tmp]# dd if=/dev/zero of=TESTDISK bs=1m count=100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 3.390829 secs (30923882 bytes/sec) > [root@neutrino /tmp]# mdconfig -a -t vnode -f ./TESTDISK > md0 > [root@neutrino /tmp]# newfs -U /dev/md0 > /dev/md0: 100.0MB (204800 sectors) block size 16384, fragment size > 2048 > using 4 cylinder groups of 25.02MB, 1601 blks, 3264 inodes. > with soft updates > super-block backups (for fsck -b #) at: > 160, 51392, 102624, 153856 > [root@neutrino /tmp]# mount /dev/md0 /mnt > [root@neutrino /tmp]# touch /mnt/foo > [root@neutrino /tmp]# mksnap_ffs /mnt/ /mnt/.snap/snap > [root@neutrino /tmp]# mdconfig -a -t vnode -f /mnt/.snap/snap > WARNING: opening backing store: /mnt/.snap/snap readonly > md1 > [root@neutrino /tmp]# mount -r /dev/md1 /mnt2 > [root@neutrino /tmp]# cd /mnt2 > [root@neutrino /mnt2]# ls -l foo > -rw-r--r-- 1 root wheel 0 Sep 21 15:37 foo > [root@neutrino /mnt2]# > > > Can you unmount /raid, and do an fsck -y /raid and make sure it's > clean? Also - are you using quotas? How big is the filesystem? > > Eric > > > > > -- > ---------------------------------------------------------------------- > -- > Eric Anderson Sr. Systems Administrator Centaur > Technology > Anything that works is better than anything that doesn't. > ---------------------------------------------------------------------- > -- From owner-freebsd-fs@FreeBSD.ORG Fri Sep 22 04:15:51 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 057D216A4A0 for ; Fri, 22 Sep 2006 04:15:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from fw.zoral.com.ua (fw.zoral.com.ua [213.186.206.134]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37C2D43D49 for ; Fri, 22 Sep 2006 04:15:49 +0000 (GMT) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id k8M4Ewnc023656 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 22 Sep 2006 07:14:58 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8) with ESMTP id k8M4Fc32009638; Fri, 22 Sep 2006 07:15:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8/Submit) id k8M4FZeQ009637; Fri, 22 Sep 2006 07:15:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 22 Sep 2006 07:15:35 +0300 From: Kostik Belousov To: Eric Anderson Message-ID: <20060922041535.GF4842@deviant.kiev.zoral.com.ua> References: <854C78DB-2099-4DA5-9E3B-F30D6947C532@jlauser.net> <4512F957.2090205@centtech.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3oCie2+XPXTnK5a5" Content-Disposition: inline In-Reply-To: <4512F957.2090205@centtech.com> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.4, clamav-milter version 0.88.4 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=1.9 required=5.0 tests=DNS_FROM_RFC_ABUSE, SPF_NEUTRAL,UNPARSEABLE_RELAY autolearn=no version=3.1.4 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.1.4 (2006-07-25) on fw.zoral.com.ua Cc: freebsd-fs@freebsd.org Subject: Re: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2006 04:15:51 -0000 --3oCie2+XPXTnK5a5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 21, 2006 at 03:43:03PM -0500, Eric Anderson wrote: > On 09/21/06 14:59, James Lauser wrote: > >Hello. > > > >I've been having some trouble with snapshots on my FreeBSD 6.1/amd64 =20 > >system. Basically, I have this system set up with a 3ware RAID card =20 > >and several disks, and use it to collect backups from my other =20 > >FreeBSD server (sparc64) and three Macs via rsync.. > > > >Every night, I have a script generate a snapshot of the RAID's file =20 > >system, and those snapshots are kept on the system for one week =20 > >before being removed (i.e. there are always 7 snapshots present on =20 > >the system), so I can recover files that were accidentally removed or = =20 > >changed. > > > >The problem is that when a large number of files are removed or =20 > >changed on the file system, the corresponding files in the snapshot =20 > >get corrupted. This, obviously, makes the snapshots quite useless. > > > >After some searching, I've found a bug report filed last year that =20 > >describes this problem exactly, though the log of that report does =20 > >not suggest that anything has been done with it. That report is at =20 > >http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/90512 > > > >Any help would be greatly appreciated. A test I ran showing the =20 > >problem, plus the output of uname -a and dmesg.boot is attached. > > > >Thanks in advance. >=20 >=20 > >Sledge# cd /raid > >Sledge# touch foo > >Sledge# ls -l foo > >-rw-r--r-- 1 root wheel 0 Sep 18 14:07 foo > >Sledge# mksnap_ffs /raid /raid/.snap/snap > >Sledge# rm foo > >Sledge# mdconfig -a -t vnode -f /raid/.snap/snap -u 4 > >WARNING: opening backing store: /raid/.snap/snap readonly > >Sledge# mount -r /dev/md4 /mnt > >Sledge# cd /mnt > >Sledge# ls -l foo > >ls: foo: Bad file descriptor > >Sledge#=20 > >Sledge#=20 > >Sledge#=20 > >Sledge# uname -a > >FreeBSD Sledge.jlauser.net 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 #4: Wed= =20 > >Sep 6 23:30:56 EDT 2006 =20 > >root@Sledge.jlauser.net:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > Hmm.. Interesting, it seems to work fine for me: >=20 > [root@neutrino /tmp]# dd if=3D/dev/zero of=3DTESTDISK bs=3D1m count=3D100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 3.390829 secs (30923882 bytes/sec) > [root@neutrino /tmp]# mdconfig -a -t vnode -f ./TESTDISK > md0 > [root@neutrino /tmp]# newfs -U /dev/md0 > /dev/md0: 100.0MB (204800 sectors) block size 16384, fragment size 2048 > using 4 cylinder groups of 25.02MB, 1601 blks, 3264 inodes. > with soft updates > super-block backups (for fsck -b #) at: > 160, 51392, 102624, 153856 > [root@neutrino /tmp]# mount /dev/md0 /mnt > [root@neutrino /tmp]# touch /mnt/foo > [root@neutrino /tmp]# mksnap_ffs /mnt/ /mnt/.snap/snap > [root@neutrino /tmp]# mdconfig -a -t vnode -f /mnt/.snap/snap > WARNING: opening backing store: /mnt/.snap/snap readonly > md1 > [root@neutrino /tmp]# mount -r /dev/md1 /mnt2 > [root@neutrino /tmp]# cd /mnt2 > [root@neutrino /mnt2]# ls -l foo > -rw-r--r-- 1 root wheel 0 Sep 21 15:37 foo > [root@neutrino /mnt2]# I think you have i386 system ? James, look at the PR/100365. Supposed fix is MFCed. Original reporter said that this changed nothing for him. I have not much time lately to look at this problem, but would like to get additional data points. BTW, use of snapshots with stock 6.1 is not very attractive idea, better to update to the 6-STABLE (many important fixes in that area were made). --3oCie2+XPXTnK5a5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFE2NnC3+MBN1Mb4gRAsu2AKCXYrVqSMteE5WsOenXE5TvFFBD/QCfZfzj +jC8TR6OFgpDSHJcYrdqGE8= =jD6O -----END PGP SIGNATURE----- --3oCie2+XPXTnK5a5-- From owner-freebsd-fs@FreeBSD.ORG Fri Sep 22 13:15:40 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DDE5616A407 for ; Fri, 22 Sep 2006 13:15:39 +0000 (UTC) (envelope-from james@jlauser.net) Received: from pancake.jlauser.net (pancake.jlauser.net [24.105.170.110]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65BA643D60 for ; Fri, 22 Sep 2006 13:15:38 +0000 (GMT) (envelope-from james@jlauser.net) Received: from [10.0.10.3] (cpe-72-226-85-167.nycap.res.rr.com [72.226.85.167]) (authenticated bits=0) by pancake.jlauser.net (8.13.6/8.13.6) with ESMTP id k8MDFaH2006268 (version=TLSv1/SSLv3 cipher=EDH-DSS-DES-CBC3-SHA bits=168 verify=NO); Fri, 22 Sep 2006 09:15:36 -0400 (EDT) (envelope-from james@jlauser.net) In-Reply-To: <20060922041535.GF4842@deviant.kiev.zoral.com.ua> References: <854C78DB-2099-4DA5-9E3B-F30D6947C532@jlauser.net> <4512F957.2090205@centtech.com> <20060922041535.GF4842@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: James Lauser Date: Fri, 22 Sep 2006 09:15:34 -0400 To: Kostik Belousov X-Mailer: Apple Mail (2.752.2) Cc: freebsd-fs@freebsd.org Subject: Re: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2006 13:15:40 -0000 Thanks a bunch for the suggestion. I upgraded overnight, and now all seems to be working. -- James L. Lauser james@jlauser.net +--------------+ | +--------+ +-+ | | + \ | | | | | | - | | | | | | | + / | | | | +--------+ + +-+ +--------------+ On Sep 22, 2006, at 0:15 , Kostik Belousov wrote: > On Thu, Sep 21, 2006 at 03:43:03PM -0500, Eric Anderson wrote: >> On 09/21/06 14:59, James Lauser wrote: >>> Hello. >>> >>> I've been having some trouble with snapshots on my FreeBSD 6.1/amd64 >>> system. Basically, I have this system set up with a 3ware RAID card >>> and several disks, and use it to collect backups from my other >>> FreeBSD server (sparc64) and three Macs via rsync.. >>> >>> Every night, I have a script generate a snapshot of the RAID's file >>> system, and those snapshots are kept on the system for one week >>> before being removed (i.e. there are always 7 snapshots present on >>> the system), so I can recover files that were accidentally >>> removed or >>> changed. >>> >>> The problem is that when a large number of files are removed or >>> changed on the file system, the corresponding files in the snapshot >>> get corrupted. This, obviously, makes the snapshots quite useless. >>> >>> After some searching, I've found a bug report filed last year that >>> describes this problem exactly, though the log of that report does >>> not suggest that anything has been done with it. That report is at >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/90512 >>> >>> Any help would be greatly appreciated. A test I ran showing the >>> problem, plus the output of uname -a and dmesg.boot is attached. >>> >>> Thanks in advance. >> >> >>> Sledge# cd /raid >>> Sledge# touch foo >>> Sledge# ls -l foo >>> -rw-r--r-- 1 root wheel 0 Sep 18 14:07 foo >>> Sledge# mksnap_ffs /raid /raid/.snap/snap >>> Sledge# rm foo >>> Sledge# mdconfig -a -t vnode -f /raid/.snap/snap -u 4 >>> WARNING: opening backing store: /raid/.snap/snap readonly >>> Sledge# mount -r /dev/md4 /mnt >>> Sledge# cd /mnt >>> Sledge# ls -l foo >>> ls: foo: Bad file descriptor >>> Sledge# >>> Sledge# >>> Sledge# >>> Sledge# uname -a >>> FreeBSD Sledge.jlauser.net 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 >>> #4: Wed >>> Sep 6 23:30:56 EDT 2006 >>> root@Sledge.jlauser.net:/usr/obj/usr/src/sys/GENERIC amd64 >> >> Hmm.. Interesting, it seems to work fine for me: >> >> [root@neutrino /tmp]# dd if=/dev/zero of=TESTDISK bs=1m count=100 >> 100+0 records in >> 100+0 records out >> 104857600 bytes transferred in 3.390829 secs (30923882 bytes/sec) >> [root@neutrino /tmp]# mdconfig -a -t vnode -f ./TESTDISK >> md0 >> [root@neutrino /tmp]# newfs -U /dev/md0 >> /dev/md0: 100.0MB (204800 sectors) block size 16384, fragment size >> 2048 >> using 4 cylinder groups of 25.02MB, 1601 blks, 3264 inodes. >> with soft updates >> super-block backups (for fsck -b #) at: >> 160, 51392, 102624, 153856 >> [root@neutrino /tmp]# mount /dev/md0 /mnt >> [root@neutrino /tmp]# touch /mnt/foo >> [root@neutrino /tmp]# mksnap_ffs /mnt/ /mnt/.snap/snap >> [root@neutrino /tmp]# mdconfig -a -t vnode -f /mnt/.snap/snap >> WARNING: opening backing store: /mnt/.snap/snap readonly >> md1 >> [root@neutrino /tmp]# mount -r /dev/md1 /mnt2 >> [root@neutrino /tmp]# cd /mnt2 >> [root@neutrino /mnt2]# ls -l foo >> -rw-r--r-- 1 root wheel 0 Sep 21 15:37 foo >> [root@neutrino /mnt2]# > I think you have i386 system ? > > James, look at the PR/100365. Supposed fix is MFCed. Original reporter > said that this changed nothing for him. I have not much time lately to > look at this problem, but would like to get additional data points. > > BTW, use of snapshots with stock 6.1 is not very attractive idea, > better > to update to the 6-STABLE (many important fixes in that area were > made). From owner-freebsd-fs@FreeBSD.ORG Sat Sep 23 09:16:39 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FB2416A57A for ; Sat, 23 Sep 2006 09:16:39 +0000 (UTC) (envelope-from mlckyam@est.it) Received: from est.it (cjm79.neoplus.adsl.tpnet.pl [83.31.62.79]) by mx1.FreeBSD.org (Postfix) with SMTP id 4599A43D46 for ; Sat, 23 Sep 2006 09:16:37 +0000 (GMT) (envelope-from mlckyam@est.it) Date: Sat, 23 Sep 2006 11:16:36 +0100 From: "emmery ailbert" To: Message-Id: <4310351723.Dcjseq-75480-96518432@est.it> MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit Cc: Subject: Urgent email get ready to get money X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Sep 2006 09:16:39 -0000 Company- A L L I A N C E E N T E R P R I S E Symbol - A E T R (Pink Sheets) C u r r e n t P r i c e: $ 0.48 Estimated Target - $ 2.00 + S T R O N G B U Y AE TR is undergoing a massive promotion and NOW is the time to get in. Add this Gem to your Watch list. Watch it Start a run monday About A L L I A N C E E N T E R P R I S E (.TaeCorp.) TaeCorp's vision is to be the recognized leader in providing Aerial Detection Systems including global de-mining, clearing a path to a safer planet for all humankind. TaeCorp's mission is to reclaim lands around the globe embedded with landmines that victimize countries and their stakeholders. N E W S TaeCorp Appoints TAM Oilfield Services as Representative in Sudan The A L L I A N C E E N T E R P R I S E Corporation ("TaeCorp") (AETR - News) is pleased to announce that it has appointed TAM Oilfield Services ("TAM"), of Khartoum, the capital of Sudan, as its representative to introduce and establish business relationships with oil and related companies interested in TaeCorp's unique landmine detection, location, mapping and removal systems and services. TAM Oilfield Services, established in 2001 under the laws and regulations of the Sudan government, has forged strong relationships with the leading petroleum companies that operate in the Sudan. "TAM's experience in the petroleum field, both with government and operating companies, will allow TaeCorp to 0ffer its services more efficiently and effectively and we are very pleased to have TAM join our team," says Ralph Proceviat, CFO of TaeCorp. Read more A C T N O W on this great news and get in before the move starts. Timber! Wait and see. Rain, rain go away; come again some other day. Your name is mud. What's done is done. Your in hot water. Run to seed. To rule the mountains is to rule the river. Read the tea leaves. Tastes like chicken. Salt of the Earth. The scythe ran into a stone. Spring forward fall back. We hung them out to dry. You can't teach an old dog new tricks.