Date: Thu, 4 Aug 2005 11:41:31 +0400 From: Dmitriy Kirhlarov <dimma@higis.ru> To: Anton Nikiforov <anton@nikiforov.ru> Cc: freebsd-cluster@freebsd.org Subject: Re: DRBD-like something for FreeBSD Message-ID: <20050804074117.GG46130@torch.higis.ru> In-Reply-To: <42F1105B.5040507@nikiforov.ru> References: <42EE559A.3080000@nikiforov.ru> <42EE7813.8020902@centtech.com> <42F1105B.5040507@nikiforov.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
--4zI0WCX1RcnW9Hbu
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi Anton!
On Wed, 03 Aug 2005, Anton Nikiforov wrote:
> not see the way to promote second node as prymary in case of real=20
> prymary crashed. Or i should test it with vrrpd (for example) and=20
> restart secondary as ggated instead of being ggatec? Did not clear yet.=
=20
> If you have the solution or an idea - could you please drop me a line?
Yes.
/usr/ports/net/freevrrpd
> algorythm looks the following for me:
> We have two computers:
> stage 0 (up and running claster)
> 1. da0 is exported via ggated over the network.
> This computer runing postgresql (for example) that storing data on da0.
> 2. da1 is being imported over network via ggatec as da1. Local da0 and=20
> remote da1 are mirrored via ggated/geom_mirror.ko
> and there is no service activity on this device. just a mirroring of data.
No.
When you work with ggate{c,d} you get devices ggate{0,1,2,...}. Not with da=
1.
> stage 1 (slave failed)
> 1. working as before
> 2. down
> Requests to update remote data failed. There should be no problem.
>=20
> stage 2 (slave is coming up)
> 1. working as before
> 2. booting and starting to mirror date from the master
> in case secondary goes down and up again there are no troubles. it just=
=20
> boots itself in the stage0 config. that is it
When slave is start you can rebuild mirror on the fly.
But it get many time (in my case) -- 54Gb slice rebuilded slow :(
> stage 3 (master is down)
> 1. down
> 2. should consider master is down and stop ggatec, then start ggated and=
=20
> export the da0. Then mount da0 and start PostgreSQL.
when master down, then freevrrpd can run script on slave machine.
> stage 4 (master coming up)
> 1. Should understand it is not the master anymore and start ggatec and=20
> mirror remote data to the local drive.
> 2. running as a master. with services moved/started at stage 3
> Here troubles begins. When bouth become synchronized the service that=20
> was running on this disk shoul be moved from 2 to 1. to do that i'll=20
> have to restart services in stage 0. But how? using vrrpd? or some other=
=20
> utility? How can i understand that the system is ready to get the=20
> Postgresql back to 1 and stop it on 2?
Yes. vrrp -- your friend. But moving from node2 -> node1, when node1 coming=
up bad idea. Service must continue to run on node2.
> stage 5 (bouth down, master coming up first)
>=20
> stage 6 (bouth down, slave is coming up first)
My algoritm.
No difference between master and slave in start time.
We have:
/etc/rc.conf -- this file identical on both nodes. Many parametes in it and=
2 includes:
/etc/rc.clh -- specific addreses and specific params for every node.
/etc/rc.cluster -- simlink dinamically switching between:
/etc/rc.custer.master or /etc/rc.cluster.slave
If we can ping shared IP, when starting -- we start as slave. In not -- mas=
ter.
We start freevrrpd with master or slave config (see /etc/rc.d/freevrrpd).
/etc/rc.d/freevrrpd run after network start, but before all daemons.
It work in RELENG_5* with new style rc-scripts.
/boot/loader.conf have different autoboot_delay on different nodes (bad gua=
rantee what important start-ping not run in one time on both machines)
> This is main confuse. Maybe i did not read mutch, but it is not clear=20
> for me, how to inderstand which drive contain the last data?
Yes. It's a problem in one case:
reboot node0 and reboot node1, when mirror not fully synchronize.
See attached scripts -- this my experiments. It have some known and, my be =
unknown issues:
- in /usr/local/ifstated/bin/ggatec-gmirror.sh script I can't get unit nam=
e with one line:
ggate=3D`/sbin/ggatec ${ggatec_flags}`
- when I restart /etc/rc.d/freevrrpd many time I get many ggate devices.
- parametert for ggate{c,d} may be not ideal, but it work on gigabit inter=
connect.
- I can't undestand what I must open on firewall for freevrrpd.
By.
Dmitriy=20
--4zI0WCX1RcnW9Hbu
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="cluster.tbz2"
Content-Transfer-Encoding: base64
QlpoOTFBWSZTWaJchtEADN1/5ey0AKB/////P///7/////4AgIEAAAEBQgAIYBH99W1d2fe9
PYAvHK27sc9d3jhrq60dnvbXT1iegBo2zyaBVK7sAcRtmEkRNRoaNTNU8p4TSekn6mppsKfq
T9FG02p6keoGTCBoP1IyeofqgZBKEBMiaNGSaZE0mynij1B4o0GmgAAaBoAB6gAONDQ0NANA
YgaAyAADTQANAMgAAAYSEiEammUbSTyBNlNNGTTI0GRpoA00BoBpoAAAEUqaNGJqm9U/T1TK
bU0Gmg9QGgaAAANAABkA0ABIkCBNAExAQamETNE09JpoyGQAxGmQAA0A/73hIeXyKDOb9nhh
3/llpLvvtcYjyEDWjSzMSJDQ2mJDMkhAoaElxENJHf5i7yXz81Il5enasHejG+RcEiUxptjb
QVMXlLYR4FWvBsaqdCJWtKTrWIm7opOtgSOoSGhJMvAYYS9O8qYY8F/7f09ZWU8dmTF6I492
ZoXLYzMxkGgw4x9w9U5CK2tcCfIeWegpg1jsSotYUC2YfvqlmH3BgP5IHIRB/VE2WrbaKWag
7KzjScsBEkObGKyhson4ZTJlWhdZRYdgiixj0vsvzdvHYv+VjLZn4r7oKyJL5hj46ucjEGSt
MjvxgFozSLeAS7QVwGkml9RHb2kqcr8uxXqixHFrQ2FRzGUj+3OFMOYyVOW0y1jvetFr3OtK
MlrhvjpNteXiY8lyrb5iQ2GULZVSGboxkQcVGK5U23jZvg5JwGs3rl5U4grkFIBy68RlbYxv
HjNiLXZdk3JXOnAiYZwrIekQcFqZbsry6U2/f0aPATq+UAyM9RcV3gs7QwJPfLuQEdmsLeTR
qNZD8S1TnE5TgHlt+tdHQdNDsVLniLx5G7JSYbuYwyzO7uNiSvTpyapJpmaCD8CfgQmgbdGw
bVYRjaR9VyCwJRHvGSBQgyEYdc98bBAOF9zIaGpW+aa5MPmX6IPQzU7muYfIbTMJHqo6bNbD
HTFX5sWa2o3qxkvHUfQep1yuHdO1RrR5IiWswcy8Zf5fg6jBRWUWJ1B4KVsIBYvEo7tdVmFl
lrTvuanVCr0wqquc/XUn2zThI1D+G58shmaCtiesDyO7OHDnxU0EPHUavRoynUzp15a+2hzc
+d4QGsyzqXETZaPXff2mdu06hazgRbUMVakXOG0ysLH1KUM3B4k9k+Sn1JQyxmStzOmLcpt0
UXDiNvAddtKPy4XlXqkVO5f0Fs3Wb05lKR254Q0MP09ciM9IvIKZuVMk4ROMnq0Ks99TsuPP
F2Vdd4taWVgTxg6w2P7cJ7u7R/pXT4PMaJvyY6LJ861vjkycLsc9U4HwfFYZrbZPwxd4Xzmq
jA8RRRS9ahvNBOG7iczuMPm9Z95AiiJJKBobUmHwl8TmRATbiEQGIaYhtiaEtd9LfQEdwDdY
f2wMksBoII7rLEiQ0ymNowYqdhUhESP1PdLUjDM6RoVMYFmrDaGMY6gzWRde9kjU3IJSS8vm
dE6JjjO3xUIkdDIwjAsoTZ3PvMTKRjld8ZeD5QYVqfK2+MxgO8HfbjcKsdDNUKc0LqfA7csb
aTWohc4SWTSO2MpM7ZgdVUYNtWnOQ3ME1yQwzqMjTAVOQrc8Kehu1DtI093mZ+XPFuwevymw
8h4zMqeUwXI9vJRmEJQDlMQR8YGf1jgvGN++cXL9rogdwYaoQBoGcbOiEGB8XlWOuNCGvl5X
GDollwjd5iRgmAq4FMxCi8znOlMV8BWFBoWSsnRZmIReaDfEwQBg7cjNepM5rQZTBk0HCd1L
PdR9Atm6BsOT1BdZsx22u7EZBSNpYkibQ3UGlAYMOm1Cr8oxhm2iDTVxM4MEkr31XJUIUOha
WIXpDAWCMCDM9G58NROugpgKyyFjWoGBIkb3fQJkiGxDCTZJtybQt/f7MeQEjnx5CRngizR5
guu85Z3M63IUmgU1VAIBS59fn1+FsdCb33rUe/56tajY8nxiuuksEG14AkNa5iQ5eTnCCs6B
wmkcqog95BXxam4kxXzJScrz18plN2DHErGYM/KSpQqgFMHTCL7umrVJuhv0GFkPsxb+QVRW
MjG/42XrBKeJW0c2YEQuA0GSJ5sDPK7FCirYWj/CiMS187o4SbRXSuhHzUnbMDVQfZNkAkiX
3QYICIOls+8HxpZK2g8IkYPflnHuzWe25p33iD1QHsyRGqxKYhEGIpFeOBCWtYhFtBJ3MxQY
0pStJTU7CR614FpzTRazCL0F82kq1Yy07XllYbQioJF94WdaIhlBjQTSRcE7gZl74RZ3XEri
cyYRcpsnATuSutuC9b8L1JmCaJAReXyTJNV0wZLkUYYuaZZZiQ1krgkNIyM0UdammMDn7hRK
QdYY02fk9LbeVEvRpfwnsfE0ms8gzBlwXdYegN4+ScykLZV5CI9qhklVI8Pxu9EenceC4WBH
Tt6sbYFRs4o5POVWRCcHxskxLiOAac2Q43QUW60dySMVCqKvUUaDemPBxuWCNc5++bQ6vs5/
xTd4q9F7ri+pIM6K9dBaydiVT9wEsTHsS79a+vm9SRHOxrsLw9Yr/XSradbZyYZXVXMKi4VF
u3c1yMRCwsK5AhthibEWJ75aVvGv3mtZuy2Eo1LFZhmnhRQsin61cU0FXIALGRVYbDr6oFSt
VFvdLKkjA21Yx6VbUq2VYlIssUCRJb4QGWw0nmIgahhw8p85BB8weM1ETeO4dg5uU5n8/d0/
CxcUzBzOtLeJpmNWrvQO07sMT5iZY3pZuYwxGh88hGWtBFG5UKBsUmozgzMiHiWZJGBL3lJY
KosMLF14UsqTL3F5pjBlPxcx/K0/CJG0EjdcvZtfBJKn0BGjQFkKjg0rdGFJyAOpAdKlBtfa
lGeBgQT8F6BfXYfZ7W1ADMmD7VRI5jEuTnxBphQUE3XuUMUxFxGJrEpq5AxpjFCWWoSGLeiz
hemGayq4Yy+xEQraxXhWwDeTyuqGG0cBKFxUvWNASE0i+Lr1sA4KRoY5bn4On4qht4nnv6yF
bjhE01zHYvEtNqGxjBsYoI4GIF4Dw7CUlf4+IGBibZL4fTB0prpQC5NaE2gAaCRpuQBBhlmv
PrtIQrUZKyDYFGTIQLr3ndcWQsARvzTJIuMB6fp9n+f9/fckef/X9rHsZe3qoVwL9Tq8vTIY
dkBzskqh50pkhs9fuD6Oa6g6o7EiuUBucQsIPWc/qY3KZuGRSsrLsHSLcZa7GdeFVCzVZSuj
ZwWwvBGnAhYnJE0VEg2FfmaV1VUwg28rbum902aazZRQ7DeToos2NCsSJvJNbd6XroRI8+HS
PejntLjslLRPiIsDLEvEJH/hIjTYCEavqcZrBi4dM5M2MmrEImPbS6nC3O2vcxFB5sIUWT2c
hUZvk6RidGLGsR41CG4YZKDbKSCEkVk1Gc4ATCegeY4nFbxZgSEYmPoDgdZIsXWUh6PaHLYZ
FEj3WrNFLhjSS0J6M2a5nQBS1cW1l1cUNe+NCRskaBFAL8/TMIOeUG0io0jJldMFzEJNDmd8
e2iqRQkRs3AjjPBHKiCp8OgYRISwhaO4EBMD8+KmIw6j5+SErTAz4IlkS6t6Jnd5drA9TlRt
C1ZZd2aqlydgVFso1Ad51rskwA5sXmlNRC5F0XyjJ1HJeeQ/SRRSRDcX1V+3W8IBraJooql0
jHUKl8zdeiikClIjcOaWs1pCwcyRfuurIxbbI1SUtTIPnfncOtcpFVbXCdKptOmZzX0hzMDp
JAa2LoIs0NiOcbiIMfBVkKnVVzAgRoGeUIH5uEH35jHYDUGz4NtMPND3DsSzVI8vmqdNCAit
j4Mmw2cheMRgzgkNQNEEjwm0qCReL1kIqUMUdaXQLK2zU33d/LqIp0PYXEtE1UitHARLd4sP
iVDlQEySKirx47cDVjL7QsMSL435g2mfP5yV4Ro4AykGt/bBI4jNaeGHIc4Za9KroAlkk5MG
x48ToQIwDMQsShrA0B1WiYdKWKG2hsQhjE8zAAzT8BUJHaMDXmLoVxJkZyrBirKQpwgLQFQd
nXkzE6ZUU21cLpzC0aktoELZKHHBJmuNxqy1t2AtYiGhptLJDRZdom0XNBxKgkU1pBU5oOHG
RDd5JGqVDT4RlkuyhzFkUMsqX/JOpObTpRyUyxE6iN4LBGkMYDTLJCVumtRup6pOgGokHQvn
xzSbBhiBBjBAQsgeiNLRoawAYoQUBKO2SmjIoUQpXHOSwAxqGMc/MsTLCwF5ZKV6CackonF1
sm8kTspGtHeDZ6CGM7SRAoYtbWvX0IhRCAGu4n1mdCh6p2d953b++wbCaQcMTKJXaikhJMJ0
SuU/BfpSXRTelKaPROGxtJjEOw/A9S4Ja+KDRr8OHa9TKJQxZeCwb1r4llwb+Zgc7UV5MJRJ
zeGFDY1wjgxjX8qwdbWsx0QUS5QEBRjLosSL4SRrFHpvKrQRsNFEY2VIUSqVVBhTCIaUkIbV
hZoKLyG08D5ZGsw21nfqC+Qut08OJkGjg0n8kIwOgKm/WtoOWwnhGrkmoGyQkMIaGxl7jabS
0DbJbepFMBoxiCwxESSopSSSJJUhQpKBjGDhtU8BicOG4ELD0mYtwKlK2mKXFcjyTScSCJKU
1Hg9cQHwpQoIan2hpwXL6XtCkdHC256D3ciyzq7J0WsLhtUHURZpI1NQ0EMXi4bd4Lia4IWa
jScjFG8m0cyQXeJpWICFDCbOBu2bWVlDGrhsRyYiJ9FsFI8JXQbl7CXMKiMWgrSBF7zoMMtp
tGDRjxrMRUUw5KQ67wodl+4v9T2eJwlzzz0JSNoXi6K6gvtI9hXIvGMGBR78JaS0PnRLC5B9
XE1DCgFl3271clNs7CD8odPNvDcIC0sTXQ8BUYwns5yEi4zbLHsSIEzahlRSi6yTGxjExjTQ
0kDZeEjwDQwpyOgmYkLmSXLbD7BYdPUdWmNs9g0Se5BKKhMmojtn07q1dYpIuZYtLS8C4M4i
6DBNDKTl41V+jJdY7gHOJTADtDSswDMsCA2hkaynBsZZgYNKMFmwRwXyQILuMljMe9QDRkMN
d76FdGelh2dFc0t+YZrQhGcklr0JEbWGrVNYi+xlvoX452oZQYo3XrL1V4uaGBGGk4HF9Wk9
3MD5Wku3WFCUAGKD0CU3zaOpMoED6rBjLh7QX9YgLDWh0hTIAuB4cMlL1GgJMZCqqUJYM9l+
AijQlQcyypoxgDqRBmx8XIHhxPvCxDvvFxXYVGdZfaCkGp8gRfms6qWkmF6odA8tjfuCA+lE
zuB4WMhkOFh4h+en2vCT5hRcKz7ZykXF2YzaoK/8XckU4UJCiXIbRA==
--4zI0WCX1RcnW9Hbu--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050804074117.GG46130>