Date: Tue, 26 Sep 2017 15:48:43 +0200 From: "O. Hartmann" <ohartmann@walstatt.org> To: Andriy Gapon <avg@FreeBSD.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r324011 - in head: cddl/contrib/opensolaris/cmd/ztest sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys sys/cddl/contrib/opensolaris/uts/... Message-ID: <20170926154843.35a405e3@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <201709261104.v8QB480K002320@repo.freebsd.org> References: <201709261104.v8QB480K002320@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 26 Sep 2017 11:04:08 +0000 (UTC) Andriy Gapon <avg@FreeBSD.org> wrote: > Author: avg > Date: Tue Sep 26 11:04:08 2017 > New Revision: 324011 > URL: https://svnweb.freebsd.org/changeset/base/324011 > > Log: > MFV r323535: 8585 improve batching done in zil_commit() > > FreeBSD notes: > - this MFV reverts FreeBSD commit r314549 to make the merge easier > - at present our emulation of cv_timedwait_hires is rather poor, > so I elected to use cv_timedwait_sbt directly > Please see the differential revision for details. > Unfortunately, I did not get any positive reviews, so there could be > bugs in the FreeBSD-specific piece of the merge. > Hence, the long MFC timeout. > > illumos/illumos-gate@1271e4b10dfaaed576c08a812f466f6e81370e5e > https://github.com/illumos/illumos-gate/commit/1271e4b10dfaaed576c08a812f466f6e81370e5e > > https://www.illumos.org/issues/8585 > The current implementation of zil_commit() can introduce significant > latency, beyond what is inherent due to the latency of the underlying > storage. The additional latency comes from two main problems: > 1. When there's outstanding ZIL blocks being written (i.e. there's > already a "writer thread" in progress), then any new calls to > zil_commit() will block waiting for the currently oustanding ZIL > blocks to complete. The blocks written for each "writer thread" is > coined a "batch", and there can only ever be a single "batch" being > written at a time. When a batch is being written, any new ZIL > transactions will have to wait for the next batch to be written, > which won't occur until the current batch finishes. > As a result, the underlying storage may not be used as efficiently > as possible. While "new" threads enter zil_commit() and are blocked > waiting for the next batch, it's possible that the underlying > storage isn't fully utilized by the current batch of ZIL blocks. In > that case, it'd be better to allow these new threads to generate > (and issue) a new ZIL block, such that it could be serviced by the > underlying storage concurrently with the other ZIL blocks that are > being serviced. > 2. Any call to zil_commit() must wait for all ZIL blocks in its "batch" > to complete, prior to zil_commit() returning. The size of any given > batch is proportional to the number of ZIL transaction in the queue > at the time that the batch starts processing the queue; which > doesn't occur until the previous batch completes. Thus, if there's a > lot of transactions in the queue, the batch could be composed of > many ZIL blocks, and each call to zil_commit() will have to wait for > all of these writes to complete (even if the thread calling > zil_commit() only cared about one of the transactions in the batch). > > Reviewed by: Brad Lewis <brad.lewis@delphix.com> > Reviewed by: Matt Ahrens <mahrens@delphix.com> > Reviewed by: George Wilson <george.wilson@delphix.com> > Approved by: Dan McDonald <danmcd@joyent.com> > Author: Prakash Surya <prakash.surya@delphix.com> > > MFC after: 1 month > Differential Revision: https://reviews.freebsd.org/D12355 > > Modified: > head/cddl/contrib/opensolaris/cmd/ztest/ztest.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil.h > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil_impl.h > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c > head/sys/cddl/contrib/opensolaris/uts/common/sys/debug.h > Directory Properties: > head/cddl/contrib/opensolaris/ (props changed) > head/sys/cddl/contrib/opensolaris/ (props changed) > > Modified: head/cddl/contrib/opensolaris/cmd/ztest/ztest.c > ============================================================================== > --- head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Sep 26 > 09:34:18 2017 (r324010) +++ > head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Sep 26 11:04:08 > 2017 (r324011) @@ -1825,13 +1825,14 @@ ztest_get_done(zgd_t *zgd, int > error) ztest_object_unlock(zd, object); > if (error == 0 && zgd->zgd_bp) > - zil_add_block(zgd->zgd_zilog, zgd->zgd_bp); > + zil_lwb_add_block(zgd->zgd_lwb, zgd->zgd_bp); > > umem_free(zgd, sizeof (*zgd)); > } > > static int > -ztest_get_data(void *arg, lr_write_t *lr, char *buf, zio_t *zio) > +ztest_get_data(void *arg, lr_write_t *lr, char *buf, struct lwb *lwb, > + zio_t *zio) > { > ztest_ds_t *zd = arg; > objset_t *os = zd->zd_os; > @@ -1845,6 +1846,10 @@ ztest_get_data(void *arg, lr_write_t *lr, char *buf, z > zgd_t *zgd; > int error; > > + ASSERT3P(lwb, !=, NULL); > + ASSERT3P(zio, !=, NULL); > + ASSERT3U(size, !=, 0); > + > ztest_object_lock(zd, object, RL_READER); > error = dmu_bonus_hold(os, object, FTAG, &db); > if (error) { > @@ -1865,7 +1870,7 @@ ztest_get_data(void *arg, lr_write_t *lr, char *buf, z > db = NULL; > > zgd = umem_zalloc(sizeof (*zgd), UMEM_NOFAIL); > - zgd->zgd_zilog = zd->zd_zilog; > + zgd->zgd_lwb = lwb; > zgd->zgd_private = zd; > > if (buf != NULL) { /* immediate write */ > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c Tue Sep > 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c Tue Sep 26 > 11:04:08 2017 (r324011) @@ -1728,6 +1728,13 @@ > dmu_sync_late_arrival(zio_t *pio, objset_t *os, dmu_sy return > (SET_ERROR(EIO)); } > + /* > + * In order to prevent the zgd's lwb from being free'd prior to > + * dmu_sync_late_arrival_done() being called, we have to ensure > + * the lwb's "max txg" takes this tx's txg into account. > + */ > + zil_lwb_add_txg(zgd->zgd_lwb, dmu_tx_get_txg(tx)); > + > dsa = kmem_alloc(sizeof (dmu_sync_arg_t), KM_SLEEP); > dsa->dsa_dr = NULL; > dsa->dsa_done = done; > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Tue > Sep 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Tue Sep > 26 11:04:08 2017 (r324011) @@ -920,7 +920,7 @@ uint64_t > dmu_tx_get_txg(dmu_tx_t *tx); > * {zfs,zvol,ztest}_get_done() args > */ > typedef struct zgd { > - struct zilog *zgd_zilog; > + struct lwb *zgd_lwb; > struct blkptr *zgd_bp; > dmu_buf_t *zgd_db; > struct rl *zgd_rl; > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil.h > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil.h Tue > Sep 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil.h Tue Sep > 26 11:04:08 2017 (r324011) @@ -40,6 +40,7 @@ extern "C" { > struct dsl_pool; > struct dsl_dataset; > +struct lwb; > > /* > * Intent log format: > @@ -140,6 +141,7 @@ typedef enum zil_create { > /* > * Intent log transaction types and record structures > */ > +#define TX_COMMIT 0 /* Commit marker (no > on-disk state) */ #define TX_CREATE 1 /* Create > file */ #define TX_MKDIR 2 /* Make directory */ > #define TX_MKXATTR 3 /* Make XATTR directory */ > @@ -388,7 +390,8 @@ typedef int zil_parse_blk_func_t(zilog_t *zilog, blkpt > typedef int zil_parse_lr_func_t(zilog_t *zilog, lr_t *lr, void *arg, > uint64_t txg); > typedef int zil_replay_func_t(); > -typedef int zil_get_data_t(void *arg, lr_write_t *lr, char *dbuf, zio_t > *zio); +typedef int zil_get_data_t(void *arg, lr_write_t *lr, char *dbuf, > + struct lwb *lwb, zio_t *zio); > > extern int zil_parse(zilog_t *zilog, zil_parse_blk_func_t *parse_blk_func, > zil_parse_lr_func_t *parse_lr_func, void *arg, uint64_t txg); > @@ -427,7 +430,8 @@ extern void zil_clean(zilog_t *zilog, uint64_t > synced_ extern int zil_suspend(const char *osname, void **cookiep); > extern void zil_resume(void *cookie); > > -extern void zil_add_block(zilog_t *zilog, const blkptr_t *bp); > +extern void zil_lwb_add_block(struct lwb *lwb, const blkptr_t *bp); > +extern void zil_lwb_add_txg(struct lwb *lwb, uint64_t txg); > extern int zil_bp_tree_add(zilog_t *zilog, const blkptr_t *bp); > > extern void zil_set_sync(zilog_t *zilog, uint64_t syncval); > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil_impl.h > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil_impl.h > Tue Sep 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zil_impl.h Tue > Sep 26 11:04:08 2017 (r324011) @@ -20,7 +20,7 @@ */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights > reserved. > - * Copyright (c) 2012 by Delphix. All rights reserved. > + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. > * Copyright (c) 2014 Integros [integros.com] > */ > > @@ -37,22 +37,76 @@ extern "C" { > #endif > > /* > - * Log write buffer. > + * Possbile states for a given lwb structure. An lwb will start out in > + * the "closed" state, and then transition to the "opened" state via a > + * call to zil_lwb_write_open(). After the lwb is "open", it can > + * transition into the "issued" state via zil_lwb_write_issue(). After > + * the lwb's zio completes, and the vdev's are flushed, the lwb will > + * transition into the "done" state via zil_lwb_write_done(), and the > + * structure eventually freed. > */ > +typedef enum { > + LWB_STATE_CLOSED, > + LWB_STATE_OPENED, > + LWB_STATE_ISSUED, > + LWB_STATE_DONE, > + LWB_NUM_STATES > +} lwb_state_t; > + > +/* > + * Log write block (lwb) > + * > + * Prior to an lwb being issued to disk via zil_lwb_write_issue(), it > + * will be protected by the zilog's "zl_writer_lock". Basically, prior > + * to it being issued, it will only be accessed by the thread that's > + * holding the "zl_writer_lock". After the lwb is issued, the zilog's > + * "zl_lock" is used to protect the lwb against concurrent access. > + */ > typedef struct lwb { > zilog_t *lwb_zilog; /* back pointer to log > struct */ blkptr_t lwb_blk; /* on disk address of this log blk > */ boolean_t lwb_slog; /* lwb_blk is on SLOG device */ > int lwb_nused; /* # used bytes in buffer */ > int lwb_sz; /* size of block and > buffer */ > + lwb_state_t lwb_state; /* the state of this lwb */ > char *lwb_buf; /* log write buffer */ > - zio_t *lwb_zio; /* zio for this buffer */ > + zio_t *lwb_write_zio; /* zio for the lwb > buffer */ > + zio_t *lwb_root_zio; /* root zio for lwb write > and flushes */ dmu_tx_t *lwb_tx; /* tx for log block allocation > */ uint64_t lwb_max_txg; /* highest txg in this lwb */ > list_node_t lwb_node; /* zilog->zl_lwb_list linkage */ > + list_t lwb_waiters; /* list of > zil_commit_waiter's */ > + avl_tree_t lwb_vdev_tree; /* vdevs to flush after lwb > write */ > + kmutex_t lwb_vdev_lock; /* protects lwb_vdev_tree */ > + hrtime_t lwb_issued_timestamp; /* when was the lwb issued? */ > } lwb_t; > > /* > + * ZIL commit waiter. > + * > + * This structure is allocated each time zil_commit() is called, and is > + * used by zil_commit() to communicate with other parts of the ZIL, such > + * that zil_commit() can know when it safe for it return. For more > + * details, see the comment above zil_commit(). > + * > + * The "zcw_lock" field is used to protect the commit waiter against > + * concurrent access. This lock is often acquired while already holding > + * the zilog's "zl_writer_lock" or "zl_lock"; see the functions > + * zil_process_commit_list() and zil_lwb_flush_vdevs_done() as examples > + * of this. Thus, one must be careful not to acquire the > + * "zl_writer_lock" or "zl_lock" when already holding the "zcw_lock"; > + * e.g. see the zil_commit_waiter_timeout() function. > + */ > +typedef struct zil_commit_waiter { > + kcondvar_t zcw_cv; /* signalled when "done" */ > + kmutex_t zcw_lock; /* protects fields of this struct */ > + list_node_t zcw_node; /* linkage in lwb_t:lwb_waiter > list */ > + lwb_t *zcw_lwb; /* back pointer to lwb when > linked */ > + boolean_t zcw_done; /* B_TRUE when "done", else > B_FALSE */ > + int zcw_zio_error; /* contains the zio > io_error value */ +} zil_commit_waiter_t; > + > +/* > * Intent log transaction lists > */ > typedef struct itxs { > @@ -94,20 +148,20 @@ struct zilog { > const zil_header_t *zl_header; /* log header buffer */ > objset_t *zl_os; /* object set we're logging */ > zil_get_data_t *zl_get_data; /* callback to get object > content */ > - zio_t *zl_root_zio; /* log writer root zio */ > + lwb_t *zl_last_lwb_opened; /* most recent lwb opened > */ > + hrtime_t zl_last_lwb_latency; /* zio latency of last lwb done > */ uint64_t zl_lr_seq; /* on-disk log record sequence number */ > uint64_t zl_commit_lr_seq; /* last committed on-disk lr seq */ > uint64_t zl_destroy_txg; /* txg of last zil_destroy() > */ uint64_t zl_replayed_seq[TXG_SIZE]; /* last replayed rec seq */ > uint64_t zl_replaying_seq; /* current replay seq number */ > uint32_t zl_suspend; /* log suspend count */ > - kcondvar_t zl_cv_writer; /* log writer thread > completion */ kcondvar_t zl_cv_suspend; /* log suspend > completion */ uint8_t zl_suspending; /* log is > currently suspending */ uint8_t zl_keep_first; /* keep > first log block in destroy */ uint8_t zl_replay; /* > replaying records while set */ uint8_t zl_stop_sync; /* > for debugging */ > - uint8_t zl_writer; /* boolean: write setup in > progress */ > + kmutex_t zl_writer_lock; /* single writer, per ZIL, at > a time */ uint8_t zl_logbias; /* latency or throughput > */ uint8_t zl_sync; /* synchronous or asynchronous */ > int zl_parse_error; /* last zil_parse() error > */ @@ -115,15 +169,10 @@ struct zilog { > uint64_t zl_parse_lr_seq; /* highest lr seq on last parse */ > uint64_t zl_parse_blk_count; /* number of blocks parsed */ > uint64_t zl_parse_lr_count; /* number of log records parsed */ > - uint64_t zl_next_batch; /* next batch number */ > - uint64_t zl_com_batch; /* committed batch number */ > - kcondvar_t zl_cv_batch[2]; /* batch condition > variables */ itxg_t zl_itxg[TXG_SIZE]; /* intent log txg > chains */ list_t zl_itx_commit_list; /* itx list to be > committed */ uint64_t zl_cur_used; /* current commit log size > used */ list_t zl_lwb_list; /* in-flight log write list > */ > - kmutex_t zl_vdev_lock; /* protects zl_vdev_tree */ > - avl_tree_t zl_vdev_tree; /* vdevs to flush in > zil_commit() */ avl_tree_t zl_bp_tree; /* track bps during log > parse */ clock_t zl_replay_time; /* lbolt of when > replay started */ uint64_t zl_replay_blks; /* number of log > blocks replayed */ @@ -131,6 +180,7 @@ struct zilog { > uint_t zl_prev_blks[ZIL_PREV_BLKS]; /* size - sector > rounded */ uint_t zl_prev_rotor; /* rotor for zl_prev[] > */ txg_node_t zl_dirty_link; /* protected by dp_dirty_zilogs > list */ > + uint64_t zl_dirty_max_txg; /* highest txg used to dirty zilog > */ }; > > typedef struct zil_bp_node { > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h Tue > Sep 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h Tue Sep > 26 11:04:08 2017 (r324011) @@ -593,6 +593,7 @@ extern enum > zio_checksum zio_checksum_dedup_select(spa extern enum zio_compress > zio_compress_select(spa_t *spa, enum zio_compress child, enum zio_compress > parent); +extern void zio_cancel(zio_t *zio); > extern void zio_suspend(spa_t *spa, zio_t *zio); > extern int zio_resume(spa_t *spa); > extern void zio_resume_wait(spa_t *spa); > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c Tue Sep > 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c Tue Sep 26 > 11:04:08 2017 (r324011) @@ -163,7 +163,7 @@ txg_fini(dsl_pool_t *dp) > tx_state_t *tx = &dp->dp_tx; int c; > > - ASSERT(tx->tx_threads == 0); > + ASSERT0(tx->tx_threads); > > mutex_destroy(&tx->tx_sync_lock); > > @@ -204,7 +204,7 @@ txg_sync_start(dsl_pool_t *dp) > > dprintf("pool %p\n", dp); > > - ASSERT(tx->tx_threads == 0); > + ASSERT0(tx->tx_threads); > > tx->tx_threads = 2; > > @@ -265,7 +265,7 @@ txg_sync_stop(dsl_pool_t *dp) > /* > * Finish off any work in progress. > */ > - ASSERT(tx->tx_threads == 2); > + ASSERT3U(tx->tx_threads, ==, 2); > > /* > * We need to ensure that we've vacated the deferred space_maps. > @@ -277,7 +277,7 @@ txg_sync_stop(dsl_pool_t *dp) > */ > mutex_enter(&tx->tx_sync_lock); > > - ASSERT(tx->tx_threads == 2); > + ASSERT3U(tx->tx_threads, ==, 2); > > tx->tx_exiting = 1; > > @@ -616,7 +616,7 @@ txg_wait_synced(dsl_pool_t *dp, uint64_t txg) > ASSERT(!dsl_pool_config_held(dp)); > > mutex_enter(&tx->tx_sync_lock); > - ASSERT(tx->tx_threads == 2); > + ASSERT3U(tx->tx_threads, ==, 2); > if (txg == 0) > txg = tx->tx_open_txg + TXG_DEFER_SIZE; > if (tx->tx_sync_txg_waiting < txg) > @@ -641,7 +641,7 @@ txg_wait_open(dsl_pool_t *dp, uint64_t txg) > ASSERT(!dsl_pool_config_held(dp)); > > mutex_enter(&tx->tx_sync_lock); > - ASSERT(tx->tx_threads == 2); > + ASSERT3U(tx->tx_threads, ==, 2); > if (txg == 0) > txg = tx->tx_open_txg + 1; > if (tx->tx_quiesce_txg_waiting < txg) > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > Tue Sep 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Tue > Sep 26 11:04:08 2017 (r324011) @@ -76,6 +76,7 @@ #include <sys/acl.h> > #include <sys/vmmeter.h> > #include <vm/vm_param.h> > +#include <sys/zil.h> > > /* > * Programming rules. > @@ -1276,7 +1277,7 @@ zfs_get_done(zgd_t *zgd, int error) > VN_RELE_ASYNC(ZTOV(zp), dsl_pool_vnrele_taskq(dmu_objset_pool(os))); > > if (error == 0 && zgd->zgd_bp) > - zil_add_block(zgd->zgd_zilog, zgd->zgd_bp); > + zil_lwb_add_block(zgd->zgd_lwb, zgd->zgd_bp); > > kmem_free(zgd, sizeof (zgd_t)); > } > @@ -1289,7 +1290,7 @@ static int zil_fault_io = 0; > * Get data to generate a TX_WRITE intent log record. > */ > int > -zfs_get_data(void *arg, lr_write_t *lr, char *buf, zio_t *zio) > +zfs_get_data(void *arg, lr_write_t *lr, char *buf, struct lwb *lwb, zio_t > *zio) { > zfsvfs_t *zfsvfs = arg; > objset_t *os = zfsvfs->z_os; > @@ -1301,8 +1302,9 @@ zfs_get_data(void *arg, lr_write_t *lr, char *buf, zio > zgd_t *zgd; > int error = 0; > > - ASSERT(zio != NULL); > - ASSERT(size != 0); > + ASSERT3P(lwb, !=, NULL); > + ASSERT3P(zio, !=, NULL); > + ASSERT3U(size, !=, 0); > > /* > * Nothing to do if the file has been removed > @@ -1320,7 +1322,7 @@ zfs_get_data(void *arg, lr_write_t *lr, char *buf, zio > } > > zgd = (zgd_t *)kmem_zalloc(sizeof (zgd_t), KM_SLEEP); > - zgd->zgd_zilog = zfsvfs->z_log; > + zgd->zgd_lwb = lwb; > zgd->zgd_private = zp; > > /* > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c Tue Sep > 26 09:34:18 2017 (r324010) +++ > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c Tue Sep 26 > 11:04:08 2017 (r324011) @@ -42,32 +42,53 @@ #include <sys/abd.h> > > /* > - * The zfs intent log (ZIL) saves transaction records of system calls > - * that change the file system in memory with enough information > - * to be able to replay them. These are stored in memory until > - * either the DMU transaction group (txg) commits them to the stable pool > - * and they can be discarded, or they are flushed to the stable log > - * (also in the pool) due to a fsync, O_DSYNC or other synchronous > - * requirement. In the event of a panic or power fail then those log > - * records (transactions) are replayed. > + * The ZFS Intent Log (ZIL) saves "transaction records" (itxs) of system > + * calls that change the file system. Each itx has enough information to > + * be able to replay them after a system crash, power loss, or > + * equivalent failure mode. These are stored in memory until either: > * > - * There is one ZIL per file system. Its on-disk (pool) format consists > - * of 3 parts: > + * 1. they are committed to the pool by the DMU transaction group > + * (txg), at which point they can be discarded; or > + * 2. they are committed to the on-disk ZIL for the dataset being > + * modified (e.g. due to an fsync, O_DSYNC, or other synchronous > + * requirement). > * > - * - ZIL header > - * - ZIL blocks > - * - ZIL records > + * In the event of a crash or power loss, the itxs contained by each > + * dataset's on-disk ZIL will be replayed when that dataset is first > + * instantianted (e.g. if the dataset is a normal fileystem, when it is > + * first mounted). > * > - * A log record holds a system call transaction. Log blocks can > - * hold many log records and the blocks are chained together. > - * Each ZIL block contains a block pointer (blkptr_t) to the next > - * ZIL block in the chain. The ZIL header points to the first > - * block in the chain. Note there is not a fixed place in the pool > - * to hold blocks. They are dynamically allocated and freed as > - * needed from the blocks available. Figure X shows the ZIL structure: > + * As hinted at above, there is one ZIL per dataset (both the in-memory > + * representation, and the on-disk representation). The on-disk format > + * consists of 3 parts: > + * > + * - a single, per-dataset, ZIL header; which points to a chain of > + * - zero or more ZIL blocks; each of which contains > + * - zero or more ZIL records > + * > + * A ZIL record holds the information necessary to replay a single > + * system call transaction. A ZIL block can hold many ZIL records, and > + * the blocks are chained together, similarly to a singly linked list. > + * > + * Each ZIL block contains a block pointer (blkptr_t) to the next ZIL > + * block in the chain, and the ZIL header points to the first block in > + * the chain. > + * > + * Note, there is not a fixed place in the pool to hold these ZIL > + * blocks; they are dynamically allocated and freed as needed from the > + * blocks available on the pool, though they can be preferentially > + * allocated from a dedicated "log" vdev. > */ > > /* > + * This controls the amount of time that a ZIL block (lwb) will remain > + * "open" when it isn't "full", and it has a thread waiting for it to be > + * committed to stable storage. Please refer to the zil_commit_waiter() > + * function (and the comments within it) for more details. > + */ > +int zfs_commit_timeout_pct = 5; > + > +/* > * Disable intent logging replay. This global ZIL switch affects all pools. > */ > int zil_replay_disable = 0; > @@ -98,6 +119,7 @@ SYSCTL_QUAD(_vfs_zfs, OID_AUTO, zil_slog_bulk, CTLFLAG > &zil_slog_bulk, 0, "Maximal SLOG commit size with sync priority"); > > static kmem_cache_t *zil_lwb_cache; > +static kmem_cache_t *zil_zcw_cache; > > #define LWB_EMPTY(lwb) ((BP_GET_LSIZE(&lwb->lwb_blk) - \ > sizeof (zil_chain_t)) == (lwb->lwb_sz - lwb->lwb_nused)) > @@ -445,6 +467,20 @@ zil_free_log_record(zilog_t *zilog, lr_t *lrc, void *t > return (0); > } > > +static int > +zil_lwb_vdev_compare(const void *x1, const void *x2) > +{ > + const uint64_t v1 = ((zil_vdev_node_t *)x1)->zv_vdev; > + const uint64_t v2 = ((zil_vdev_node_t *)x2)->zv_vdev; > + > + if (v1 < v2) > + return (-1); > + if (v1 > v2) > + return (1); > + > + return (0); > +} > + > static lwb_t * > zil_alloc_lwb(zilog_t *zilog, blkptr_t *bp, boolean_t slog, uint64_t txg) > { > @@ -454,10 +490,13 @@ zil_alloc_lwb(zilog_t *zilog, blkptr_t *bp, boolean_t > lwb->lwb_zilog = zilog; > lwb->lwb_blk = *bp; > lwb->lwb_slog = slog; > + lwb->lwb_state = LWB_STATE_CLOSED; > lwb->lwb_buf = zio_buf_alloc(BP_GET_LSIZE(bp)); > lwb->lwb_max_txg = txg; > - lwb->lwb_zio = NULL; > + lwb->lwb_write_zio = NULL; > + lwb->lwb_root_zio = NULL; > lwb->lwb_tx = NULL; > + lwb->lwb_issued_timestamp = 0; > if (BP_GET_CHECKSUM(bp) == ZIO_CHECKSUM_ZILOG2) { > lwb->lwb_nused = sizeof (zil_chain_t); > lwb->lwb_sz = BP_GET_LSIZE(bp); > @@ -470,9 +509,54 @@ zil_alloc_lwb(zilog_t *zilog, blkptr_t *bp, boolean_t > list_insert_tail(&zilog->zl_lwb_list, lwb); > mutex_exit(&zilog->zl_lock); > > + ASSERT(!MUTEX_HELD(&lwb->lwb_vdev_lock)); > + ASSERT(avl_is_empty(&lwb->lwb_vdev_tree)); > + ASSERT(list_is_empty(&lwb->lwb_waiters)); > + > return (lwb); > } > > +static void > +zil_free_lwb(zilog_t *zilog, lwb_t *lwb) > +{ > + ASSERT(MUTEX_HELD(&zilog->zl_lock)); > + ASSERT(!MUTEX_HELD(&lwb->lwb_vdev_lock)); > + ASSERT(list_is_empty(&lwb->lwb_waiters)); > + > + if (lwb->lwb_state == LWB_STATE_OPENED) { > + avl_tree_t *t = &lwb->lwb_vdev_tree; > + void *cookie = NULL; > + zil_vdev_node_t *zv; > + > + while ((zv = avl_destroy_nodes(t, &cookie)) != NULL) > + kmem_free(zv, sizeof (*zv)); > + > + ASSERT3P(lwb->lwb_root_zio, !=, NULL); > + ASSERT3P(lwb->lwb_write_zio, !=, NULL); > + > + zio_cancel(lwb->lwb_root_zio); > + zio_cancel(lwb->lwb_write_zio); > + > + lwb->lwb_root_zio = NULL; > + lwb->lwb_write_zio = NULL; > + } else { > + ASSERT3S(lwb->lwb_state, !=, LWB_STATE_ISSUED); > + } > + > + ASSERT(avl_is_empty(&lwb->lwb_vdev_tree)); > + ASSERT3P(lwb->lwb_write_zio, ==, NULL); > + ASSERT3P(lwb->lwb_root_zio, ==, NULL); > + > + /* > + * Clear the zilog's field to indicate this lwb is no longer > + * valid, and prevent use-after-free errors. > + */ > + if (zilog->zl_last_lwb_opened == lwb) > + zilog->zl_last_lwb_opened = NULL; > + > + kmem_cache_free(zil_lwb_cache, lwb); > +} > + > /* > * Called when we create in-memory log transactions so that we know > * to cleanup the itxs at the end of spa_sync(). > @@ -483,12 +567,16 @@ zilog_dirty(zilog_t *zilog, uint64_t txg) > dsl_pool_t *dp = zilog->zl_dmu_pool; > dsl_dataset_t *ds = dmu_objset_ds(zilog->zl_os); > > + ASSERT(spa_writeable(zilog->zl_spa)); > + > if (ds->ds_is_snapshot) > panic("dirtying snapshot!"); > > if (txg_list_add(&dp->dp_dirty_zilogs, zilog, txg)) { > /* up the hold count until we can be written out */ > dmu_buf_add_ref(ds->ds_dbuf, zilog); > + > + zilog->zl_dirty_max_txg = MAX(txg, zilog->zl_dirty_max_txg); > } > } > > @@ -556,7 +644,7 @@ zil_create(zilog_t *zilog) > */ > if (BP_IS_HOLE(&blk) || BP_SHOULD_BYTESWAP(&blk)) { > tx = dmu_tx_create(zilog->zl_os); > - VERIFY(dmu_tx_assign(tx, TXG_WAIT) == 0); > + VERIFY0(dmu_tx_assign(tx, TXG_WAIT)); > dsl_dataset_dirty(dmu_objset_ds(zilog->zl_os), tx); > txg = dmu_tx_get_txg(tx); > > @@ -573,7 +661,7 @@ zil_create(zilog_t *zilog) > } > > /* > - * Allocate a log write buffer (lwb) for the first log block. > + * Allocate a log write block (lwb) for the first log block. > */ > if (error == 0) > lwb = zil_alloc_lwb(zilog, &blk, slog, txg); > @@ -594,13 +682,13 @@ zil_create(zilog_t *zilog) > } > > /* > - * In one tx, free all log blocks and clear the log header. > - * If keep_first is set, then we're replaying a log with no content. > - * We want to keep the first block, however, so that the first > - * synchronous transaction doesn't require a txg_wait_synced() > - * in zil_create(). We don't need to txg_wait_synced() here either > - * when keep_first is set, because both zil_create() and zil_destroy() > - * will wait for any in-progress destroys to complete. > + * In one tx, free all log blocks and clear the log header. If keep_first > + * is set, then we're replaying a log with no content. We want to keep the > + * first block, however, so that the first synchronous transaction doesn't > + * require a txg_wait_synced() in zil_create(). We don't need to > + * txg_wait_synced() here either when keep_first is set, because both > + * zil_create() and zil_destroy() will wait for any in-progress destroys > + * to complete. > */ > void > zil_destroy(zilog_t *zilog, boolean_t keep_first) > @@ -621,7 +709,7 @@ zil_destroy(zilog_t *zilog, boolean_t keep_first) > return; > > tx = dmu_tx_create(zilog->zl_os); > - VERIFY(dmu_tx_assign(tx, TXG_WAIT) == 0); > + VERIFY0(dmu_tx_assign(tx, TXG_WAIT)); > dsl_dataset_dirty(dmu_objset_ds(zilog->zl_os), tx); > txg = dmu_tx_get_txg(tx); > > @@ -638,8 +726,8 @@ zil_destroy(zilog_t *zilog, boolean_t keep_first) > list_remove(&zilog->zl_lwb_list, lwb); > if (lwb->lwb_buf != NULL) > zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); > - zio_free_zil(zilog->zl_spa, txg, &lwb->lwb_blk); > - kmem_cache_free(zil_lwb_cache, lwb); > + zio_free(zilog->zl_spa, txg, &lwb->lwb_blk); > + zil_free_lwb(zilog, lwb); > } > } else if (!keep_first) { > zil_destroy_sync(zilog, tx); > @@ -777,24 +865,64 @@ zil_check_log_chain(dsl_pool_t *dp, dsl_dataset_t *ds, > return ((error == ECKSUM || error == ENOENT) ? 0 : error); > } > > -static int > -zil_vdev_compare(const void *x1, const void *x2) > +/* > + * When an itx is "skipped", this function is used to properly mark the > + * waiter as "done, and signal any thread(s) waiting on it. An itx can > + * be skipped (and not committed to an lwb) for a variety of reasons, > + * one of them being that the itx was committed via spa_sync(), prior to > + * it being committed to an lwb; this can happen if a thread calling > + * zil_commit() is racing with spa_sync(). > + */ > +static void > +zil_commit_waiter_skip(zil_commit_waiter_t *zcw) > { > - const uint64_t v1 = ((zil_vdev_node_t *)x1)->zv_vdev; > - const uint64_t v2 = ((zil_vdev_node_t *)x2)->zv_vdev; > + mutex_enter(&zcw->zcw_lock); > + ASSERT3B(zcw->zcw_done, ==, B_FALSE); > + zcw->zcw_done = B_TRUE; > + cv_broadcast(&zcw->zcw_cv); > + mutex_exit(&zcw->zcw_lock); > +} > > - if (v1 < v2) > - return (-1); > - if (v1 > v2) > - return (1); > +/* > + * This function is used when the given waiter is to be linked into an > + * lwb's "lwb_waiter" list; i.e. when the itx is committed to the lwb. > + * At this point, the waiter will no longer be referenced by the itx, > + * and instead, will be referenced by the lwb. > + */ > +static void > +zil_commit_waiter_link_lwb(zil_commit_waiter_t *zcw, lwb_t *lwb) > +{ > + mutex_enter(&zcw->zcw_lock); > + ASSERT(!list_link_active(&zcw->zcw_node)); > + ASSERT3P(zcw->zcw_lwb, ==, NULL); > + ASSERT3P(lwb, !=, NULL); > + ASSERT(lwb->lwb_state == LWB_STATE_OPENED || > + lwb->lwb_state == LWB_STATE_ISSUED); > > - return (0); > + list_insert_tail(&lwb->lwb_waiters, zcw); > + zcw->zcw_lwb = lwb; > + mutex_exit(&zcw->zcw_lock); > } > > +/* > + * This function is used when zio_alloc_zil() fails to allocate a ZIL > + * block, and the given waiter must be linked to the "nolwb waiters" > + * list inside of zil_process_commit_list(). > + */ > +static void > +zil_commit_waiter_link_nolwb(zil_commit_waiter_t *zcw, list_t *nolwb) > +{ > + mutex_enter(&zcw->zcw_lock); > + ASSERT(!list_link_active(&zcw->zcw_node)); > + ASSERT3P(zcw->zcw_lwb, ==, NULL); > + list_insert_tail(nolwb, zcw); > + mutex_exit(&zcw->zcw_lock); > +} > + > void > -zil_add_block(zilog_t *zilog, const blkptr_t *bp) > +zil_lwb_add_block(lwb_t *lwb, const blkptr_t *bp) > { > - avl_tree_t *t = &zilog->zl_vdev_tree; > + avl_tree_t *t = &lwb->lwb_vdev_tree; > avl_index_t where; > zil_vdev_node_t *zv, zvsearch; > int ndvas = BP_GET_NDVAS(bp); > @@ -803,14 +931,7 @@ zil_add_block(zilog_t *zilog, const blkptr_t *bp) > if (zfs_nocacheflush) > return; > > - ASSERT(zilog->zl_writer); > - > - /* > - * Even though we're zl_writer, we still need a lock because the > - * zl_get_data() callbacks may have dmu_sync() done callbacks > - * that will run concurrently. > - */ > - mutex_enter(&zilog->zl_vdev_lock); > + mutex_enter(&lwb->lwb_vdev_lock); > for (i = 0; i < ndvas; i++) { > zvsearch.zv_vdev = DVA_GET_VDEV(&bp->blk_dva[i]); > if (avl_find(t, &zvsearch, &where) == NULL) { > @@ -819,59 +940,117 @@ zil_add_block(zilog_t *zilog, const blkptr_t *bp) > avl_insert(t, zv, where); > } > } > - mutex_exit(&zilog->zl_vdev_lock); > + mutex_exit(&lwb->lwb_vdev_lock); > } > > +void > +zil_lwb_add_txg(lwb_t *lwb, uint64_t txg) > +{ > + lwb->lwb_max_txg = MAX(lwb->lwb_max_txg, txg); > +} > + > +/* > + * This function is a called after all VDEVs associated with a given lwb > + * write have completed their DKIOCFLUSHWRITECACHE command; or as soon > + * as the lwb write completes, if "zfs_nocacheflush" is set. > + * > + * The intention is for this function to be called as soon as the > + * contents of an lwb are considered "stable" on disk, and will survive > + * any sudden loss of power. At this point, any threads waiting for the > + * lwb to reach this state are signalled, and the "waiter" structures > + * are marked "done". > + */ > static void > -zil_flush_vdevs(zilog_t *zilog) > +zil_lwb_flush_vdevs_done(zio_t *zio) > { > - spa_t *spa = zilog->zl_spa; > - avl_tree_t *t = &zilog->zl_vdev_tree; > - void *cookie = NULL; > - zil_vdev_node_t *zv; > - zio_t *zio = NULL; > + lwb_t *lwb = zio->io_private; > + zilog_t *zilog = lwb->lwb_zilog; > + dmu_tx_t *tx = lwb->lwb_tx; > + zil_commit_waiter_t *zcw; > > - ASSERT(zilog->zl_writer); > + spa_config_exit(zilog->zl_spa, SCL_STATE, lwb); > > + zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); > + > + mutex_enter(&zilog->zl_lock); > + > /* > - * We don't need zl_vdev_lock here because we're the zl_writer, > - * and all zl_get_data() callbacks are done. > + * Ensure the lwb buffer pointer is cleared before releasing the > + * txg. If we have had an allocation failure and the txg is > + * waiting to sync then we want zil_sync() to remove the lwb so > + * that it's not picked up as the next new one in > + * zil_process_commit_list(). zil_sync() will only remove the > + * lwb if lwb_buf is null. > */ > - if (avl_numnodes(t) == 0) > - return; > + lwb->lwb_buf = NULL; > + lwb->lwb_tx = NULL; > > - spa_config_enter(spa, SCL_STATE, FTAG, RW_READER); > + ASSERT3U(lwb->lwb_issued_timestamp, >, 0); > + zilog->zl_last_lwb_latency = gethrtime() - lwb->lwb_issued_timestamp; > > - while ((zv = avl_destroy_nodes(t, &cookie)) != NULL) { > - vdev_t *vd = vdev_lookup_top(spa, zv->zv_vdev); > - if (vd != NULL && !vd->vdev_nowritecache) { > - if (zio == NULL) > - zio = zio_root(spa, NULL, NULL, > ZIO_FLAG_CANFAIL); > - zio_flush(zio, vd); > - } > - kmem_free(zv, sizeof (*zv)); > + lwb->lwb_root_zio = NULL; > + lwb->lwb_state = LWB_STATE_DONE; > + > + if (zilog->zl_last_lwb_opened == lwb) { > + /* > + * Remember the highest committed log sequence number > + * for ztest. We only update this value when all the log > + * writes succeeded, because ztest wants to ASSERT that > + * it got the whole log chain. > + */ > + zilog->zl_commit_lr_seq = zilog->zl_lr_seq; > } > > + while ((zcw = list_head(&lwb->lwb_waiters)) != NULL) { > + mutex_enter(&zcw->zcw_lock); > + > + ASSERT(list_link_active(&zcw->zcw_node)); > + list_remove(&lwb->lwb_waiters, zcw); > + > + ASSERT3P(zcw->zcw_lwb, ==, lwb); > + zcw->zcw_lwb = NULL; > + > + zcw->zcw_zio_error = zio->io_error; > + > + ASSERT3B(zcw->zcw_done, ==, B_FALSE); > + zcw->zcw_done = B_TRUE; > + cv_broadcast(&zcw->zcw_cv); > + > + mutex_exit(&zcw->zcw_lock); > + } > + > + mutex_exit(&zilog->zl_lock); > + > /* > - * Wait for all the flushes to complete. Not all devices actually > - * support the DKIOCFLUSHWRITECACHE ioctl, so it's OK if it fails. > + * Now that we've written this log block, we have a stable pointer > + * to the next block in the chain, so it's OK to let the txg in > + * which we allocated the next block sync. > */ > - if (zio) > - (void) zio_wait(zio); > - > - spa_config_exit(spa, SCL_STATE, FTAG); > + dmu_tx_commit(tx); > } > > /* > - * Function called when a log block write completes > + * This is called when an lwb write completes. This means, this specific > + * lwb was written to disk, and all dependent lwb have also been > + * written to disk. > + * > + * At this point, a DKIOCFLUSHWRITECACHE command hasn't been issued to > + * the VDEVs involved in writing out this specific lwb. The lwb will be > + * "done" once zil_lwb_flush_vdevs_done() is called, which occurs in the > + * zio completion callback for the lwb's root zio. > */ > static void > zil_lwb_write_done(zio_t *zio) > { > lwb_t *lwb = zio->io_private; > + spa_t *spa = zio->io_spa; > zilog_t *zilog = lwb->lwb_zilog; > - dmu_tx_t *tx = lwb->lwb_tx; > + avl_tree_t *t = &lwb->lwb_vdev_tree; > + void *cookie = NULL; > + zil_vdev_node_t *zv; > > + ASSERT3S(spa_config_held(spa, SCL_STATE, RW_READER), !=, 0); > + > ASSERT(BP_GET_COMPRESS(zio->io_bp) == ZIO_COMPRESS_OFF); > ASSERT(BP_GET_TYPE(zio->io_bp) == DMU_OT_INTENT_LOG); > ASSERT(BP_GET_LEVEL(zio->io_bp) == 0); > @@ -880,58 +1059,115 @@ zil_lwb_write_done(zio_t *zio) > ASSERT(!BP_IS_HOLE(zio->io_bp)); > ASSERT(BP_GET_FILL(zio->io_bp) == 0); > > - /* > - * Ensure the lwb buffer pointer is cleared before releasing > - * the txg. If we have had an allocation failure and > - * the txg is waiting to sync then we want want zil_sync() > - * to remove the lwb so that it's not picked up as the next new > - * one in zil_commit_writer(). zil_sync() will only remove > - * the lwb if lwb_buf is null. > - */ > abd_put(zio->io_abd); > - zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); > + > + ASSERT3S(lwb->lwb_state, ==, LWB_STATE_ISSUED); > + > mutex_enter(&zilog->zl_lock); > - lwb->lwb_buf = NULL; > - lwb->lwb_tx = NULL; > + lwb->lwb_write_zio = NULL; > mutex_exit(&zilog->zl_lock); > > + if (avl_numnodes(t) == 0) > + return; > + > /* > - * Now that we've written this log block, we have a stable pointer > - * to the next block in the chain, so it's OK to let the txg in > - * which we allocated the next block sync. > + * If there was an IO error, we're not going to call zio_flush() > + * on these vdevs, so we simply empty the tree and free the > + * nodes. We avoid calling zio_flush() since there isn't any > + * good reason for doing so, after the lwb block failed to be > + * written out. > */ > - dmu_tx_commit(tx); > + if (zio->io_error != 0) { > + while ((zv = avl_destroy_nodes(t, &cookie)) != NULL) > + kmem_free(zv, sizeof (*zv)); > + return; > + } > + > + while ((zv = avl_destroy_nodes(t, &cookie)) != NULL) { > + vdev_t *vd = vdev_lookup_top(spa, zv->zv_vdev); > + if (vd != NULL) > + zio_flush(lwb->lwb_root_zio, vd); > + kmem_free(zv, sizeof (*zv)); > + } > } > > /* > - * Initialize the io for a log block. > + * This function's purpose is to "open" an lwb such that it is ready to > + * accept new itxs being committed to it. To do this, the lwb's zio > + * structures are created, and linked to the lwb. This function is > + * idempotent; if the passed in lwb has already been opened, this > + * function is essentially a no-op. > */ > static void > -zil_lwb_write_init(zilog_t *zilog, lwb_t *lwb) > +zil_lwb_write_open(zilog_t *zilog, lwb_t *lwb) > { > zbookmark_phys_t zb; > zio_priority_t prio; > > + ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); > + ASSERT3P(lwb, !=, NULL); > + EQUIV(lwb->lwb_root_zio == NULL, lwb->lwb_state == LWB_STATE_CLOSED); > + EQUIV(lwb->lwb_root_zio != NULL, lwb->lwb_state == LWB_STATE_OPENED); > + > SET_BOOKMARK(&zb, lwb->lwb_blk.blk_cksum.zc_word[ZIL_ZC_OBJSET], > ZB_ZIL_OBJECT, ZB_ZIL_LEVEL, > lwb->lwb_blk.blk_cksum.zc_word[ZIL_ZC_SEQ]); > > - if (zilog->zl_root_zio == NULL) { > - zilog->zl_root_zio = zio_root(zilog->zl_spa, NULL, NULL, > - ZIO_FLAG_CANFAIL); > - } > - if (lwb->lwb_zio == NULL) { > + if (lwb->lwb_root_zio == NULL) { > abd_t *lwb_abd = abd_get_from_buf(lwb->lwb_buf, > BP_GET_LSIZE(&lwb->lwb_blk)); > + > if (!lwb->lwb_slog || zilog->zl_cur_used <= zil_slog_bulk) > prio = ZIO_PRIORITY_SYNC_WRITE; > else > prio = ZIO_PRIORITY_ASYNC_WRITE; > - lwb->lwb_zio = zio_rewrite(zilog->zl_root_zio, zilog->zl_spa, > - 0, &lwb->lwb_blk, lwb_abd, BP_GET_LSIZE(&lwb->lwb_blk), > - zil_lwb_write_done, lwb, prio, > - ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE, &zb); > + > + lwb->lwb_root_zio = zio_root(zilog->zl_spa, > + zil_lwb_flush_vdevs_done, lwb, ZIO_FLAG_CANFAIL); > + ASSERT3P(lwb->lwb_root_zio, !=, NULL); > + > + lwb->lwb_write_zio = zio_rewrite(lwb->lwb_root_zio, > + zilog->zl_spa, 0, &lwb->lwb_blk, lwb_abd, > + BP_GET_LSIZE(&lwb->lwb_blk), zil_lwb_write_done, lwb, > + prio, ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE, &zb); > + ASSERT3P(lwb->lwb_write_zio, !=, NULL); > + > + lwb->lwb_state = LWB_STATE_OPENED; > + > + mutex_enter(&zilog->zl_lock); > + > + /* > + * The zilog's "zl_last_lwb_opened" field is used to > + * build the lwb/zio dependency chain, which is used to > + * preserve the ordering of lwb completions that is > + * required by the semantics of the ZIL. Each new lwb > + * zio becomes a parent of the "previous" lwb zio, such > + * that the new lwb's zio cannot complete until the > + * "previous" lwb's zio completes. > + * > + * This is required by the semantics of zil_commit(); > + * the commit waiters attached to the lwbs will be woken > + * in the lwb zio's completion callback, so this zio > + * dependency graph ensures the waiters are woken in the > + * correct order (the same order the lwbs were created). > + */ > + lwb_t *last_lwb_opened = zilog->zl_last_lwb_opened; > + if (last_lwb_opened != NULL && > + last_lwb_opened->lwb_state != LWB_STATE_DONE) { > + ASSERT(last_lwb_opened->lwb_state == > LWB_STATE_OPENED || > + last_lwb_opened->lwb_state == LWB_STATE_ISSUED); > + ASSERT3P(last_lwb_opened->lwb_root_zio, !=, NULL); > + zio_add_child(lwb->lwb_root_zio, > + last_lwb_opened->lwb_root_zio); > + } > + zilog->zl_last_lwb_opened = lwb; > + > + mutex_exit(&zilog->zl_lock); > } > + > + ASSERT3P(lwb->lwb_root_zio, !=, NULL); > + ASSERT3P(lwb->lwb_write_zio, !=, NULL); > + ASSERT3S(lwb->lwb_state, ==, LWB_STATE_OPENED); > } > > /* > @@ -953,7 +1189,7 @@ uint64_t zil_block_buckets[] = { > * Calls are serialized. > */ > static lwb_t * > -zil_lwb_write_start(zilog_t *zilog, lwb_t *lwb, boolean_t last) > +zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) > { > lwb_t *nlwb = NULL; > zil_chain_t *zilc; > @@ -965,6 +1201,11 @@ zil_lwb_write_start(zilog_t *zilog, lwb_t *lwb, boolea > int i, error; > > *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** > _______________________________________________ > svn-src-head@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-head > To unsubscribe, send any mail to "svn-src-head-unsubscribe@freebsd.org" Build world/kernel on r324015 fails due to: [...] ===> lib/libpam/modules/pam_login_access (obj) --- cddl/lib__L --- --- zil.o --- /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2333:19: warning: implicit declaration of function 'cv_timedwait_sbt' is invalid in C99 [-Wimplicit-function-declaration] int wait_err = cv_timedwait_sbt(&zcw->zcw_cv, ^ /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2335:8: error: use of undeclared identifier 'C_ABSOLUTE' C_ABSOLUTE); ^ --- lib__L --- --- obj_subdir_lib/libpam/modules/pam_nologin --- ===> lib/libpam/modules/pam_nologin (obj) --- cddl/lib__L --- 1 warning and 1 error generated.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170926154843.35a405e3>