Date: Wed, 8 Nov 2017 16:16:42 -0700 From: markham breitbach <markham_breitbach@ssimicro.com> To: freebsd-fs@freebsd.org Subject: glusterfs volume status not showing online Message-ID: <ac6dbc8c-1628-a396-3020-941a2330c1b1@ssimicro.com>
next in thread | raw e-mail | index | archive | help
Hi There, I hope this is the right place for this question.=A0 I am trying to setup glusterfs on FreeBSD. I originally did some testing a while back with glusterfs 3.9.0_2 on FreeBSD 10.3 and a run through the quickstart seemed to work OK on a couple of small VMs. I am now trying to do some testing on bare metal using FreeBSD 11.1 (and now 10.4). I installed the pkg and ran the "quickstart." but now I find that `gluster volume status` doesn't seem to be working (currently on glusterfs-3.11.1_2) There doesn't seem to be anything particularly telling in glusterd.log, and I can actually mount the volume and use it. Any advice on where to look next? Thanks, -Markham ---- quick start ---- =A0# pkg install glusterfs =A0# zfs create -omountpoint=3D/groot zroot/groot =A0# service glusterd onestart =A0# gluster peer probe gtest4 =A0# gluster volume create apple replica 2 gtest4:/groot/apple gtest3:/groot/apple =A0# gluster volume start apple =A0# gluster volume status Status of volume: apple Gluster process=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 TCP Port=A0 RDMA Port=A0 Online=A0 Pid -------------------------------------------------------------------------= ----- Brick gtest4:/groot/apple=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 N/A=A0=A0=A0=A0=A0=A0 N/A=A0=A0=A0=A0=A0=A0=A0 N=A0=A0=A0=A0=A0=A0= N/A Brick gtest3:/groot/apple=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 N/A=A0=A0=A0=A0=A0=A0 N/A=A0=A0=A0=A0=A0=A0=A0 N=A0=A0=A0=A0=A0=A0= N/A Self-heal Daemon on localhost=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 N= /A=A0=A0=A0=A0=A0=A0 N/A=A0=A0=A0=A0=A0=A0=A0 N=A0=A0=A0=A0=A0=A0 967 Self-heal Daemon on gtest4=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 N/A=A0=A0=A0=A0=A0=A0 N/A=A0=A0=A0=A0=A0=A0=A0 N=A0=A0=A0=A0=A0=A0 97= 0 Task Status of Volume apple -------------------------------------------------------------------------= ----- There are no active volume tasks ----- glusterd.log ----- [2017-11-08 22:29:09.132427] I [MSGID: 100030] [glusterfsd.c:2476:main] 0-/usr/local/sbin/glusterd: Started running /usr/local/sbin/glusterd version 3.11.1 (args: /usr/local/sbin/glusterd --pid-file=3D/var/run/glusterd.pid) [2017-11-08 22:29:09.138608] I [MSGID: 106478] [glusterd.c:1422:init] 0-management: Maximum allowed open file descriptors set to 65536 [2017-11-08 22:29:09.138676] I [MSGID: 106479] [glusterd.c:1469:init] 0-management: Using /var/db/glusterd as working directory [2017-11-08 22:29:09.143727] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: Cannot open "/usr/local/lib/glusterfs/3.11.1/rpc-transport/rdma.so" [2017-11-08 22:29:09.143758] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine [2017-11-08 22:29:09.143777] W [rpcsvc.c:1660:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed [2017-11-08 22:29:09.143796] E [MSGID: 106243] [glusterd.c:1693:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport [2017-11-08 22:29:09.147373] E [MSGID: 101032] [store.c:433:gf_store_handle_retrieve] 0-: Path corresponding to /var/db/glusterd/glusterd.info. [No such file or directory] [2017-11-08 22:29:09.147414] E [MSGID: 101032] [store.c:433:gf_store_handle_retrieve] 0-: Path corresponding to /var/db/glusterd/glusterd.info. [No such file or directory] [2017-11-08 22:29:09.147416] I [MSGID: 106514] [glusterd-store.c:2215:glusterd_restore_op_version] 0-management: Detected new install. Setting op-version to maximum : 31100 [2017-11-08 22:29:09.147477] E [MSGID: 101032] [store.c:433:gf_store_handle_retrieve] 0-: Path corresponding to /var/db/glusterd/options. [No such file or directory] [2017-11-08 22:29:09.168706] I [MSGID: 106194] [glusterd-store.c:3772:glusterd_store_retrieve_missed_snaps_list] 0-management: No missed snaps list. Final graph: +------------------------------------------------------------------------= ------+ =A0 1: volume management =A0 2:=A0=A0=A0=A0 type mgmt/glusterd =A0 3:=A0=A0=A0=A0 option rpc-auth.auth-glusterfs on =A0 4:=A0=A0=A0=A0 option rpc-auth.auth-unix on =A0 5:=A0=A0=A0=A0 option rpc-auth.auth-null on =A0 6:=A0=A0=A0=A0 option rpc-auth-allow-insecure on =A0 7:=A0=A0=A0=A0 option transport.socket.listen-backlog 128 =A0 8:=A0=A0=A0=A0 option event-threads 1 =A0 9:=A0=A0=A0=A0 option ping-timeout 0 =A010:=A0=A0=A0=A0 option transport.socket.read-fail-log off =A011:=A0=A0=A0=A0 option transport.socket.keepalive-interval 2 =A012:=A0=A0=A0=A0 option transport.socket.keepalive-time 10 =A013:=A0=A0=A0=A0 option transport-type rdma =A014:=A0=A0=A0=A0 option working-directory /var/db/glusterd =A015: end-volume =A016: +------------------------------------------------------------------------= ------+ [2017-11-08 22:29:41.245073] I [MSGID: 106487] [glusterd-handler.c:1242:__glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req gtest4 24007 [2017-11-08 22:29:41.269548] I [MSGID: 106129] [glusterd-handler.c:3623:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: gtest4 (24007) [2017-11-08 22:29:41.310764] W [MSGID: 106062] [glusterd-handler.c:3399:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout [2017-11-08 22:29:41.310813] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2017-11-08 22:29:41.310890] W [MSGID: 101002] [options.c:954:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction [2017-11-08 22:29:41.317314] I [MSGID: 106498] [glusterd-handler.c:3549:glusterd_friend_add] 0-management: connect returned 0 [2017-11-08 22:29:41.318182] E [MSGID: 101032] [store.c:433:gf_store_handle_retrieve] 0-: Path corresponding to /var/db/glusterd/glusterd.info. [No such file or directory] [2017-11-08 22:29:41.318258] I [MSGID: 106477] [glusterd.c:190:glusterd_uuid_generate_save] 0-management: generated UUID: cee1794a-5e83-415e-88e0-eae018f9e745 [2017-11-08 22:29:41.426904] I [MSGID: 106511] [glusterd-rpc-ops.c:261:__glusterd_probe_cbk] 0-management: Received probe resp from uuid: 2dcbff56-88e8-48c8-814a-101f3d32c2b3, host: gtest4 [2017-11-08 22:29:41.426950] I [MSGID: 106511] [glusterd-rpc-ops.c:421:__glusterd_probe_cbk] 0-glusterd: Received resp to probe req [2017-11-08 22:29:41.455508] I [MSGID: 106493] [glusterd-rpc-ops.c:485:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2dcbff56-88e8-48c8-814a-101f3d32c2b3, host: gtest4, port: = 0 [2017-11-08 22:29:41.496977] I [MSGID: 106163] [glusterd-handshake.c:1309:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31100 [2017-11-08 22:29:41.521751] I [MSGID: 106490] [glusterd-handler.c:2890:__glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 2dcbff56-88e8-48c8-814a-101f3d32c2b3 [2017-11-08 22:29:41.525628] I [MSGID: 106493] [glusterd-handler.c:2953:__glusterd_handle_probe_query] 0-glusterd: Responded to gtest4.ssimicro.com, op_ret: 0, op_errno: 0, ret: 0 [2017-11-08 22:29:41.538527] I [MSGID: 106490] [glusterd-handler.c:2539:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2dcbff56-88e8-48c8-814a-101f3d32c2b= 3 [2017-11-08 22:29:41.557101] I [MSGID: 106493] [glusterd-handler.c:3799:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to gtest4 (0), ret: 0, op_ret: 0 [2017-11-08 22:29:41.574502] I [MSGID: 106492] [glusterd-handler.c:2717:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2dcbff56-88e8-48c8-814a-101f3d32c2b3 [2017-11-08 22:29:41.574567] I [MSGID: 106502] [glusterd-handler.c:2762:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend [2017-11-08 22:29:41.574634] I [MSGID: 106493] [glusterd-rpc-ops.c:700:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2dcbff56-88e8-48c8-814a-101f3d32c2b3 [2017-11-08 22:30:47.866210] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: Cannot open "/usr/local/lib/glusterfs/3.11.1/xlator/nfs/server.so" [2017-11-08 22:30:55.267120] I [MSGID: 106143] [glusterd-pmap.c:279:pmap_registry_bind] 0-pmap: adding brick /groot/apple on port 49152 [2017-11-08 22:30:55.267548] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2017-11-08 22:30:55.346373] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600 [2017-11-08 22:30:55.346761] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600 [2017-11-08 22:30:55.346875] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped [2017-11-08 22:30:55.346924] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: nfs service is stopped [2017-11-08 22:30:55.346968] I [MSGID: 106600] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed [2017-11-08 22:30:55.347136] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600 [2017-11-08 22:30:55.347911] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped [2017-11-08 22:30:55.347950] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: glustershd service is stopped [2017-11-08 22:30:55.348012] I [MSGID: 106567] [glusterd-svc-mgmt.c:196:glusterd_svc_start] 0-management: Starting glustershd service [2017-11-08 22:30:56.365516] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600 [2017-11-08 22:30:56.365837] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600 [2017-11-08 22:30:56.366019] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped [2017-11-08 22:30:56.366071] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: bitd service is stopped [2017-11-08 22:30:56.366235] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600 [2017-11-08 22:30:56.366372] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped [2017-11-08 22:30:56.366409] I [MSGID: 106568] [glusterd-svc-mgmt.c:228:glusterd_svc_stop] 0-management: scrub service is stopped [2017-11-08 22:30:57.707989] E [run.c:190:runner_log] (-->0x803ecad43 <notify+0xb1de3> at /usr/local/lib/glusterfs/3.11.1/xlator/mgmt/glusterd.so -->0x8008a6504 <runner_log+0x104> at /usr/local/lib/libglusterfs.so.0 ) 0-management: Failed to execute script: /var/db/glusterd/hooks/1/start/post/S29CTDBsetup.sh --volname=3Dapple --first=3Dyes --version=3D1 --volume-op=3Dstart --gd-workdir=3D/var/db/gl= usterd [2017-11-08 22:30:57.722114] E [run.c:190:runner_log] (-->0x803ecad43 <notify+0xb1de3> at /usr/local/lib/glusterfs/3.11.1/xlator/mgmt/glusterd.so -->0x8008a6504 <runner_log+0x104> at /usr/local/lib/libglusterfs.so.0 ) 0-management: Failed to execute script: /var/db/glusterd/hooks/1/start/post/S30samba-start.sh --volname=3Dapple --first=3Dyes --version=3D1 --volume-op=3Dstart --gd-workdir=3D/var/db/gl= usterd [2017-11-08 22:31:01.537697] I [MSGID: 106499] [glusterd-handler.c:4302:__glusterd_handle_status_volume] 0-management: Received status volume req for volume apple [2017-11-08 22:31:05.749436] I [MSGID: 106499] [glusterd-handler.c:4302:__glusterd_handle_status_volume] 0-management: Received status volume req for volume apple [2017-11-08 22:31:55.783261] I [MSGID: 106499] [glusterd-handler.c:4302:__glusterd_handle_status_volume] 0-management: Received status volume req for volume apple [2017-11-08 22:34:59.660782] I [MSGID: 106487] [glusterd-handler.c:1484:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ac6dbc8c-1628-a396-3020-941a2330c1b1>