Safe Haskell	Safe-Inferred
Language	GHC2021

Database.LSMTree.Internal.RunReader

Contents

Exported for WriteBufferReader

Description

A run that is being read incrementally.

Synopsis

data RunReader m h = RunReader {
- readerCurrentPage :: !(MutVar (PrimState m) (Maybe RawPage))
- readerCurrentEntryNo :: !(PrimVar (PrimState m) Word16)
- readerKOpsHandle :: !(Handle h)
- readerBlobFile :: !(Ref (BlobFile m h))
- readerRunDataCaching :: !RunDataCaching
- readerHasFS :: !(HasFS m h)
- readerHasBlockIO :: !(HasBlockIO m h)
}
data OffsetKey
- = NoOffsetKey
- | OffsetKey !SerialisedKey
new :: forall m h. (MonadMask m, MonadSTM m, PrimMonad m) => OffsetKey -> Ref (Run m h) -> m (RunReader m h)
next :: forall m h. (MonadMask m, MonadSTM m, MonadST m) => RunReader m h -> m (Result m h)
close :: (MonadSTM m, MonadMask m, PrimMonad m) => RunReader m h -> m ()
data Result m h
- = Empty
- | ReadEntry !SerialisedKey !(Entry m h)
data Entry m h
- = Entry !(Entry SerialisedValue (RawBlobRef m h))
- | EntryOverflow !(Entry SerialisedValue (RawBlobRef m h)) !RawPage !Word32 ![RawOverflowPage]
toFullEntry :: Entry m h -> Entry SerialisedValue (RawBlobRef m h)
appendOverflow :: Word32 -> [RawOverflowPage] -> SerialisedValue -> SerialisedValue
mkEntryOverflow :: Entry SerialisedValue (RawBlobRef m h) -> RawPage -> Word32 -> [RawOverflowPage] -> Entry m h
readDiskPage :: (MonadCatch m, PrimMonad m) => HasFS m h -> Handle h -> m (Maybe RawPage)
readOverflowPages :: (MonadSTM m, MonadThrow m, PrimMonad m) => HasFS m h -> Handle h -> Word32 -> m [RawOverflowPage]

Documentation

data RunReader m h Source #

Allows reading the k/ops of a run incrementally, using its own read-only file handle and in-memory cache of the current disk page.

Creating a RunReader does not retain a reference to the Run, but does retain an independent reference on the run's blob file. It is not necessary to separately retain the Run for correct use of the RunReader. There is one important caveat however: the RunReader maintains the validity of BlobRefs only up until the point where the reader is drained (or explicitly closed). In particular this means BlobRefs can be invalidated as soon as the next returns Empty. If this is not sufficient then it is necessary to separately retain a reference to the Run or its BlobFile to preserve the validity of BlobRefs.

New pages are loaded when trying to read their first entry.

TODO(optimise): Reuse page buffers using some kind of allocator. However, deciding how long a page needs to stay around is not trivial.

Constructors

RunReader

Fields

readerCurrentPage :: !(MutVar (PrimState m) (Maybe RawPage))
The disk page currently being read. If it is Nothing, the reader is considered closed.
readerCurrentEntryNo :: !(PrimVar (PrimState m) Word16)
The index of the entry to be returned by the next call to next.
readerKOpsHandle :: !(Handle h)
Read mode file handle into the run's k/ops file. We rely on it to track the position of the next disk page to read, instead of keeping a counter ourselves. Also, the run's handle is supposed to be opened with O_DIRECT, which is counterproductive here.
readerBlobFile :: !(Ref (BlobFile m h))
The blob file from the run this reader is reading from.
readerRunDataCaching :: !RunDataCaching
readerHasFS :: !(HasFS m h)
readerHasBlockIO :: !(HasBlockIO m h)

data OffsetKey Source #

Constructors

NoOffsetKey
OffsetKey !SerialisedKey

new :: forall m h. (MonadMask m, MonadSTM m, PrimMonad m) => OffsetKey -> Ref (Run m h) -> m (RunReader m h) Source #

next :: forall m h. (MonadMask m, MonadSTM m, MonadST m) => RunReader m h -> m (Result m h) Source #

Stop using the RunReader after getting Empty, because the Reader is automatically closed!

close :: (MonadSTM m, MonadMask m, PrimMonad m) => RunReader m h -> m () Source #

This function should be called when discarding a RunReader before it was done (i.e. returned Empty). This avoids leaking file handles. Once it has been called, do not use the reader any more!

data Result m h Source #

The SerialisedKey and SerialisedValue point into the in-memory disk page. Keeping them alive will also prevent garbage collection of the 4k byte array, so if they're long-lived, consider making a copy!

Constructors

Empty
ReadEntry !SerialisedKey !(Entry m h)

data Entry m h Source #

Constructors

Entry !(Entry SerialisedValue (RawBlobRef m h))
EntryOverflow	A large entry. The caller might be interested in various different (redundant) representation, so we return all of them.
Fields !(Entry SerialisedValue (RawBlobRef m h)) The value is just a prefix, with the remainder in the overflow pages. !RawPage A page containing the single entry (or rather its prefix). !Word32 Non-zero length of the overflow in bytes. ![RawOverflowPage] The overflow pages containing the suffix of the value (so at least the number of bytes specified above). TODO(optimise): Sometimes, reading the overflow pages is not necessary. We could just return the page index and offer a separate function to do the disk I/O once needed.

toFullEntry :: Entry m h -> Entry SerialisedValue (RawBlobRef m h) Source #

appendOverflow :: Word32 -> [RawOverflowPage] -> SerialisedValue -> SerialisedValue Source #

Exported for WriteBufferReader

mkEntryOverflow :: Entry SerialisedValue (RawBlobRef m h) -> RawPage -> Word32 -> [RawOverflowPage] -> Entry m h Source #

readDiskPage :: (MonadCatch m, PrimMonad m) => HasFS m h -> Handle h -> m (Maybe RawPage) Source #

Returns Nothing on EOF.

readOverflowPages :: (MonadSTM m, MonadThrow m, PrimMonad m) => HasFS m h -> Handle h -> Word32 -> m [RawOverflowPage] Source #

Throws exception on EOF. If a suffix was expected, the file should have it. Reads full pages, despite the suffix only using part of the last page.