Safe Haskell | Safe-Inferred |
---|---|
Language | GHC2021 |
Database.LSMTree.Internal.IncomingRun
Contents
Synopsis
- data IncomingRun m h
- = Single !(Ref (Run m h))
- | Merging !MergePolicyForLevel !NominalDebt !(PrimVar (PrimState m) NominalCredits) !(Ref (MergingRun LevelMergeType m h))
- data MergePolicyForLevel
- duplicateIncomingRun :: (PrimMonad m, MonadMask m) => IncomingRun m h -> m (IncomingRun m h)
- releaseIncomingRun :: (PrimMonad m, MonadMask m) => IncomingRun m h -> m ()
- newIncomingSingleRun :: (PrimMonad m, MonadThrow m) => Ref (Run m h) -> m (IncomingRun m h)
- newIncomingMergingRun :: (PrimMonad m, MonadThrow m) => MergePolicyForLevel -> NominalDebt -> Ref (MergingRun LevelMergeType m h) -> m (IncomingRun m h)
- snapshotIncomingRun :: PrimMonad m => IncomingRun m h -> m (Either (Ref (Run m h)) (MergePolicyForLevel, NominalDebt, NominalCredits, Ref (MergingRun LevelMergeType m h)))
- newtype NominalDebt = NominalDebt Int
- newtype NominalCredits = NominalCredits Int
- nominalDebtAsCredits :: NominalDebt -> NominalCredits
- supplyCreditsIncomingRun :: (MonadSTM m, MonadST m, MonadMVar m, MonadMask m) => TableConfig -> LevelNo -> IncomingRun m h -> NominalCredits -> m ()
- immediatelyCompleteIncomingRun :: (MonadSTM m, MonadST m, MonadMVar m, MonadMask m) => TableConfig -> LevelNo -> IncomingRun m h -> m (Ref (Run m h))
Documentation
data IncomingRun m h Source #
An incoming run is either a single run, or a merge.
Constructors
Single !(Ref (Run m h)) | |
Merging !MergePolicyForLevel !NominalDebt !(PrimVar (PrimState m) NominalCredits) !(Ref (MergingRun LevelMergeType m h)) |
data MergePolicyForLevel Source #
Constructors
LevelTiering | |
LevelLevelling |
Instances
Show MergePolicyForLevel Source # | |
Defined in Database.LSMTree.Internal.IncomingRun Methods showsPrec :: Int -> MergePolicyForLevel -> ShowS # show :: MergePolicyForLevel -> String # showList :: [MergePolicyForLevel] -> ShowS # | |
NFData MergePolicyForLevel Source # | |
Defined in Database.LSMTree.Internal.IncomingRun Methods rnf :: MergePolicyForLevel -> () # | |
Eq MergePolicyForLevel Source # | |
Defined in Database.LSMTree.Internal.IncomingRun Methods (==) :: MergePolicyForLevel -> MergePolicyForLevel -> Bool # (/=) :: MergePolicyForLevel -> MergePolicyForLevel -> Bool # | |
DecodeVersioned MergePolicyForLevel Source # | |
Defined in Database.LSMTree.Internal.Snapshot.Codec Methods decodeVersioned :: SnapshotVersion -> Decoder s MergePolicyForLevel Source # | |
Encode MergePolicyForLevel Source # | |
Defined in Database.LSMTree.Internal.Snapshot.Codec Methods encode :: MergePolicyForLevel -> Encoding Source # |
duplicateIncomingRun :: (PrimMonad m, MonadMask m) => IncomingRun m h -> m (IncomingRun m h) Source #
releaseIncomingRun :: (PrimMonad m, MonadMask m) => IncomingRun m h -> m () Source #
newIncomingSingleRun :: (PrimMonad m, MonadThrow m) => Ref (Run m h) -> m (IncomingRun m h) Source #
newIncomingMergingRun :: (PrimMonad m, MonadThrow m) => MergePolicyForLevel -> NominalDebt -> Ref (MergingRun LevelMergeType m h) -> m (IncomingRun m h) Source #
snapshotIncomingRun :: PrimMonad m => IncomingRun m h -> m (Either (Ref (Run m h)) (MergePolicyForLevel, NominalDebt, NominalCredits, Ref (MergingRun LevelMergeType m h))) Source #
Credits and credit tracking
With scheduled merges, each update (e.g., insert) on a table contributes to the progression of ongoing merges in the levels structure. This ensures that merges are finished in time before a new merge has to be started. The points in the evolution of the levels structure where new merges are started are known: a flush of a full write buffer will create a new run on the first level, and after sufficient flushes (e.g., 4) we will start at least one new merge on the second level. This may cascade down to lower levels depending on how full the levels are. As such, we have a well-defined measure to determine when merges should be finished: it only depends on the maximum size of the write buffer!
The simplest solution to making sure merges are done in time is to step them to completion immediately when started. This does not, however, spread out work over time nicely. Instead, we schedule merge work based on how many updates are made on the table, taking care to ensure that the merge is finished just in time before the next flush comes around, and not too early.
The progression is tracked using nominal credits. Each individual update contributes a single credit to each level, since each level contains precisely one ongoing merge. Contributing a credit does not, however, translate directly to performing one unit of merging work:
- The amount of work to do for one credit is adjusted depending on the actual
size of the merge we are doing. Last-level merges, for example, can have
larger inputs, and therefore we have to do a little more work for each
credit. Or input runs involved in a merge can be less than maximal size for
the level, and so there may be less merging work to do. As such, we scale
NominalCredits
toMergeCredits
, and then supply theMergeCredits
to theMergingRun
. - Supplying
MergeCredits
to aMergingRun
does not necessarily directly translate into performing merging work. Merge credits are accumulated until they go over a threshold, after which a batch of merge work will be performed. Configuring this threshold should allow a good balance between spreading out I/O and achieving good (concurrent) performance.
Merging runs can be shared across tables, which means that multiple threads
can contribute to the same merge concurrently. Incoming runs however are not
shared between tables. As such the tracking of NominalCredits
does not need
to use any concurrency precautions.
newtype NominalDebt Source #
Total merge debt to complete the merge in an incoming run.
This corresponds to the number (worst case, minimum number) of update operations inserted into the table, before we will expect the merge to complete.
Constructors
NominalDebt Int |
Instances
NFData NominalDebt Source # | |
Defined in Database.LSMTree.Internal.IncomingRun Methods rnf :: NominalDebt -> () # | |
Eq NominalDebt Source # | |
Defined in Database.LSMTree.Internal.IncomingRun | |
DecodeVersioned NominalDebt Source # | |
Defined in Database.LSMTree.Internal.Snapshot.Codec Methods decodeVersioned :: SnapshotVersion -> Decoder s NominalDebt Source # | |
Encode NominalDebt Source # | |
Defined in Database.LSMTree.Internal.Snapshot.Codec Methods encode :: NominalDebt -> Encoding Source # |
newtype NominalCredits Source #
Merge credits that get supplied to a table's levels.
This corresponds to the number of update operations inserted into the table.
Constructors
NominalCredits Int |
Instances
supplyCreditsIncomingRun :: (MonadSTM m, MonadST m, MonadMVar m, MonadMask m) => TableConfig -> LevelNo -> IncomingRun m h -> NominalCredits -> m () Source #
Supply a given number of nominal credits to the merge in an incoming run. This is a relative addition of credits, not a new absolute total value.
immediatelyCompleteIncomingRun :: (MonadSTM m, MonadST m, MonadMVar m, MonadMask m) => TableConfig -> LevelNo -> IncomingRun m h -> m (Ref (Run m h)) Source #
Supply enough credits to complete the merge now.