| Safe Haskell | Safe-Inferred |
|---|---|
| Language | GHC2021 |
Database.LSMTree.Internal.Readers
Contents
Description
Multiple inputs (write buffers, runs) that are being read incrementally.
Synopsis
- data Readers m h = Readers {
- readersHeap :: !(MutableHeap (PrimState m) (ReadCtx m h))
- readersNext :: !(MutVar (PrimState m) (ReadCtx m h))
- data OffsetKey
- data ReaderSource m h
- = FromWriteBuffer !WriteBuffer !(Ref (WriteBufferBlobs m h))
- | FromRun !(Ref (Run m h))
- | FromReaders !ReadersMergeType ![ReaderSource m h]
- data ReadersMergeType
- new :: forall m h. (MonadMask m, MonadST m, MonadSTM m) => ResolveSerialisedValue -> OffsetKey -> [ReaderSource m h] -> m (Maybe (Readers m h))
- close :: (MonadMask m, MonadSTM m, PrimMonad m) => Readers m h -> m ()
- peekKey :: PrimMonad m => Readers m h -> m SerialisedKey
- data HasMore
- pop :: (MonadMask m, MonadSTM m, MonadST m) => ResolveSerialisedValue -> Readers m h -> m (SerialisedKey, Entry m h, HasMore)
- dropWhileKey :: (MonadMask m, MonadSTM m, MonadST m) => ResolveSerialisedValue -> Readers m h -> SerialisedKey -> m (Int, HasMore)
- data Reader m h
- = ReadBuffer !(MutVar (PrimState m) [KOp m h])
- | ReadRun !(RunReader m h)
- | ReadReaders !ReadersMergeType !(SMaybe (Readers m h))
- newtype ReaderNumber = ReaderNumber Int
- data ReadCtx m h = ReadCtx {
- readCtxHeadKey :: !SerialisedKey
- readCtxHeadEntry :: !(Entry m h)
- readCtxNumber :: !ReaderNumber
- readCtxReader :: !(Reader m h)
Documentation
A collection of runs and write buffers being read from, yielding elements in order. More precisely, that means first ordered by their key, then by the input run they came from. This is important for resolving multiple entries with the same key into one.
Construct with new, then keep calling pop.
If aborting early, remember to call close!
Creating a Readers does not retain a reference to the input Runs or the
WriteBufferBlobs, but does retain an independent reference on their blob
files. It is not necessary to separately retain the Runs or the
WriteBufferBlobs for correct use of the Readers. There is one important
caveat however: to preserve the validity of BlobRefs then it is necessary
to separately retain a reference to the Run or its BlobFile to preserve
the validity of BlobRefs.
TODO: do this more nicely by changing Reader to preserve the BlobFile
ref until it is explicitly closed, and also retain the BlobFile from the
WBB and release all of these BlobFiles once the Readers is itself closed.
Constructors
| Readers | |
Fields
| |
Constructors
| NoOffsetKey | |
| OffsetKey !SerialisedKey |
data ReaderSource m h Source #
Constructors
| FromWriteBuffer !WriteBuffer !(Ref (WriteBufferBlobs m h)) | |
| FromRun !(Ref (Run m h)) | |
| FromReaders !ReadersMergeType ![ReaderSource m h] | Recursive case, allowing to build a tree of readers for a merging tree. |
data ReadersMergeType Source #
Constructors
| MergeLevel | |
| MergeUnion |
Instances
| Show ReadersMergeType Source # | |
Defined in Database.LSMTree.Internal.Readers Methods showsPrec :: Int -> ReadersMergeType -> ShowS # show :: ReadersMergeType -> String # showList :: [ReadersMergeType] -> ShowS # | |
| Eq ReadersMergeType Source # | |
Defined in Database.LSMTree.Internal.Readers Methods (==) :: ReadersMergeType -> ReadersMergeType -> Bool # (/=) :: ReadersMergeType -> ReadersMergeType -> Bool # | |
new :: forall m h. (MonadMask m, MonadST m, MonadSTM m) => ResolveSerialisedValue -> OffsetKey -> [ReaderSource m h] -> m (Maybe (Readers m h)) Source #
close :: (MonadMask m, MonadSTM m, PrimMonad m) => Readers m h -> m () Source #
Clean up the resources held by the readers.
Only call this function when aborting before all readers have been drained!
peekKey :: PrimMonad m => Readers m h -> m SerialisedKey Source #
Return the smallest key present in the readers, without consuming any entries.
pop :: (MonadMask m, MonadSTM m, MonadST m) => ResolveSerialisedValue -> Readers m h -> m (SerialisedKey, Entry m h, HasMore) Source #
Remove the entry with the smallest key and return it. If there are multiple
entries with that key, it removes the one from the source that came first
in list supplied to new. No resolution of multiple entries takes place.
Arguments
| :: (MonadMask m, MonadSTM m, MonadST m) | |
| => ResolveSerialisedValue | |
| -> Readers m h | |
| -> SerialisedKey | |
| -> m (Int, HasMore) | How many were dropped? |
Drop all entries with a key that is smaller or equal to the supplied one.
Internals
An individual reader must be able to produce a sequence of pairs of
SerialisedKey and Entry, with ordered und unique keys.
TODO: This is slightly inelegant. This module could work generally for anything that can produce elements, but currently is very specific to having write buffer and run readers. Also, for run merging, no write buffer is involved, but we still need to branch on this sum type. A more general version is possible, but despite SPECIALISE-ing everything showed ~100 bytes of extra allocations per entry that is read (which might be avoidable with some tinkering).
Constructors
| ReadBuffer !(MutVar (PrimState m) [KOp m h]) | The list allows to incrementally read from the write buffer without having to find the next entry in the Map again (requiring key comparisons) or having to copy out all entries. TODO: more efficient representation? benchmark! |
| ReadRun !(RunReader m h) | |
| ReadReaders !ReadersMergeType !(SMaybe (Readers m h)) | Recursively read from another reader. This requires keeping track of
its We represent the recursive reader and TODO: maybe it's a slightly more ergonomic alternative to no close the
|
newtype ReaderNumber Source #
Constructors
| ReaderNumber Int |
Instances
| Eq ReaderNumber Source # | |
Defined in Database.LSMTree.Internal.Readers | |
| Ord ReaderNumber Source # | |
Defined in Database.LSMTree.Internal.Readers Methods compare :: ReaderNumber -> ReaderNumber -> Ordering # (<) :: ReaderNumber -> ReaderNumber -> Bool # (<=) :: ReaderNumber -> ReaderNumber -> Bool # (>) :: ReaderNumber -> ReaderNumber -> Bool # (>=) :: ReaderNumber -> ReaderNumber -> Bool # max :: ReaderNumber -> ReaderNumber -> ReaderNumber # min :: ReaderNumber -> ReaderNumber -> ReaderNumber # | |
Each heap element needs some more context than just the reader.
E.g. the Eq instance we need to be able to access the first key to be read
in a pure way.
TODO(optimisation): We allocate this record for each k/op. This might be avoidable, see ideas below.
Constructors
| ReadCtx | |
Fields
| |
Instances
| Eq (ReadCtx m h) Source # | |
| Ord (ReadCtx m h) Source # | Makes sure we resolve entries in the right order. |
Defined in Database.LSMTree.Internal.Readers | |