lsm-tree-0.1.0.0: Log-structured merge-trees
Safe HaskellSafe-Inferred
LanguageGHC2021

Database.LSMTree.Internal

Description

This module brings together the internal parts to provide an API in terms of untyped serialised keys, values and blobs. It makes no distinction between normal and monoidal tables, accepting both blobs and mupserts. The typed normal and monoidal APIs then provide more type-safe wrappers and handle serialisation.

Apart from defining the API, this module mainly deals with concurrency- and exception-safe opening and closing of resources. Any other non-trivial logic should live somewhere else.

Synopsis

Existentials

data Session' m Source #

Constructors

forall h.Typeable h => Session' !(Session m h) 

Instances

Instances details
NFData (Session' m) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: Session' m -> () #

data Table' m k v b Source #

Constructors

forall h.Typeable h => Table' (Table m h) 

Instances

Instances details
NFData (Table' m k v b) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: Table' m k v b -> () #

data Cursor' m k v b Source #

Constructors

forall h.Typeable h => Cursor' (Cursor m h) 

Instances

Instances details
NFData (Cursor' m k v b) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: Cursor' m k v b -> () #

data NormalTable m k v b Source #

Constructors

forall h.Typeable h => NormalTable !(Table m h) 

Instances

Instances details
NFData (NormalTable m k v b) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: NormalTable m k v b -> () #

data NormalCursor m k v b Source #

Constructors

forall h.Typeable h => NormalCursor !(Cursor m h) 

Instances

Instances details
NFData (NormalCursor m k v b) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: NormalCursor m k v b -> () #

data MonoidalTable m k v Source #

Constructors

forall h.Typeable h => MonoidalTable !(Table m h) 

Instances

Instances details
NFData (MonoidalTable m k v) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: MonoidalTable m k v -> () #

data MonoidalCursor m k v Source #

Constructors

forall h.Typeable h => MonoidalCursor !(Cursor m h) 

Instances

Instances details
NFData (MonoidalCursor m k v) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: MonoidalCursor m k v -> () #

Exceptions

data TableUnionNotCompatibleError Source #

A table union was constructed with two tables that are not compatible.

Constructors

ErrTableUnionHandleTypeMismatch 

Fields

  • !Int

    The index of the first table.

  • !TypeRep

    The type of the filesystem handle of the first table.

  • !Int

    The index of the second table.

  • !TypeRep

    The type of the filesystem handle of the second table.

ErrTableUnionSessionMismatch 

Fields

  • !Int

    The index of the first table.

  • !FsErrorPath

    The session directory of the first table.

  • !Int

    The index of the second table.

  • !FsErrorPath

    The session directory of the second table.

data SnapshotNotCompatibleError Source #

The named snapshot is not compatible with the expected type.

Constructors

ErrSnapshotWrongTableType

The named snapshot is not compatible with the current public API module. For instance, the snapshot was created using the simple API, but was opened using the full API.

Fields

ErrSnapshotWrongLabel

The named snapshot is not compatible with the given label.

Fields

data BlobRefInvalidError Source #

A BlobRef used with retrieveBlobs was invalid.

BlobRefs are obtained from lookups in a Table, but they may be invalidated by subsequent changes in that Table. In general the reliable way to retrieve blobs is not to change the Table before retrieving the blobs. To allow later retrievals, duplicate the table before making modifications and keep the table open until all blob retrievals are complete.

Constructors

ErrBlobRefInvalid !Int

The Int index indicates the first invalid BlobRef.

data FileCorruptedError Source #

The file is corrupted.

Constructors

ErrFileFormatInvalid

The file fails to parse.

Fields

ErrFileChecksumMismatch

The file CRC32 checksum is invalid.

Fields

Tracing

Session

data Session m h Source #

A session provides context that is shared across multiple tables.

For more information, see Session.

Constructors

Session 

Fields

Instances

Instances details
NFData (Session m h) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: Session m h -> () #

data SessionEnv m h Source #

Constructors

SessionEnv 

Fields

withOpenSession :: (MonadSTM m, MonadThrow m) => Session m h -> (SessionEnv m h -> m a) -> m a Source #

withOpenSession ensures that the session stays open for the duration of the provided continuation.

NOTE: any operation except sessionClose can use this function.

Implementation of public API

withSession :: (MonadMask m, MonadSTM m, MonadMVar m, PrimMonad m) => Tracer m LSMTreeTrace -> HasFS m h -> HasBlockIO m h -> FsPath -> (Session m h -> m a) -> m a Source #

openSession Source #

Arguments

:: forall m h. (MonadSTM m, MonadMVar m, PrimMonad m, MonadMask m) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> FsPath

Path to the session directory

-> m (Session m h) 

closeSession :: (MonadMask m, MonadSTM m, MonadMVar m, PrimMonad m) => Session m h -> m () Source #

See closeSession.

A session's global resources will only be released once it is sure that no tables or cursors are open anymore.

Table

data Table m h Source #

A handle to an on-disk key/value table.

For more information, see Table.

Constructors

Table 

Fields

Instances

Instances details
NFData (Table m h) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: Table m h -> () #

data TableState m h Source #

A table may assume that its corresponding session is still open as long as the table is open. A session's global resources, and therefore resources that are inherited by the table, will only be released once the session is sure that no tables are open anymore.

Constructors

TableOpen !(TableEnv m h) 
TableClosed 

data TableEnv m h Source #

Constructors

TableEnv 

Fields

  • tableSessionEnv :: !(SessionEnv m h)

    Use this instead of tableSession for easy access. An open table may assume that its session is open.

  • tableContent :: !(RWVar m (TableContent m h))

    All of the state being in a single StrictMVar is a relatively simple solution, but there could be more concurrency. For example, while inserts are in progress, lookups could still look at the old state without waiting for the MVar.

    TODO: switch to more fine-grained synchronisation approach

withOpenTable :: (MonadSTM m, MonadThrow m) => Table m h -> (TableEnv m h -> m a) -> m a Source #

withOpenTable ensures that the table stays open for the duration of the provided continuation.

NOTE: any operation except close can use this function.

Implementation of public API

type ResolveSerialisedValue = SerialisedValue -> SerialisedValue -> SerialisedValue Source #

Value resolve function: what to do when resolving two Mupdates

withTable :: (MonadMask m, MonadSTM m, MonadMVar m, PrimMonad m) => Session m h -> TableConfig -> (Table m h -> m a) -> m a Source #

new :: (MonadSTM m, MonadMVar m, PrimMonad m, MonadMask m) => Session m h -> TableConfig -> m (Table m h) Source #

See new.

close :: (MonadMask m, MonadSTM m, MonadMVar m, PrimMonad m) => Table m h -> m () Source #

See close.

rangeLookup Source #

Arguments

:: (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) 
=> ResolveSerialisedValue 
-> Range SerialisedKey 
-> Table m h 
-> (SerialisedKey -> SerialisedValue -> Maybe (WeakBlobRef m h) -> res)

How to map to a query result, different for normal/monoidal

-> m (Vector res) 

updates :: (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) => ResolveSerialisedValue -> Vector (SerialisedKey, Entry SerialisedValue SerialisedBlob) -> Table m h -> m () Source #

See updates.

Does not enforce that mupsert and blobs should not occur in the same table.

Cursor API

data Cursor m h Source #

A read-only view into the table state at the time of cursor creation.

For more information, see Cursor.

The representation of a cursor is similar to that of a Table, but simpler, as it is read-only.

Constructors

Cursor 

Fields

Instances

Instances details
NFData (Cursor m h) Source # 
Instance details

Defined in Database.LSMTree.Internal

Methods

rnf :: Cursor m h -> () #

data CursorState m h Source #

Constructors

CursorOpen !(CursorEnv m h) 
CursorClosed

Calls to a closed cursor raise an exception.

data CursorEnv m h Source #

Constructors

CursorEnv 

Fields

  • cursorSession :: !(Session m h)

    The session that this cursor belongs to.

    NOTE: Consider using the cursorSessionEnv field instead of acquiring the session lock.

  • cursorSessionEnv :: !(SessionEnv m h)

    Use this instead of cursorSession for easy access. An open cursor may assume that its session is open. A session's global resources, and therefore resources that are inherited by the cursor, will only be released once the session is sure that no cursors are open anymore.

  • cursorId :: !CursorId

    Session-unique identifier for this cursor.

  • cursorReaders :: !(Maybe (Readers m h))

    Readers are immediately discarded once they are Drained, so if there is a Just, we can assume that it has further entries. Note that, while the readers do retain references on the blob files while they are active, once they are drained they do not. This could invalidate any BlobRefs previously handed out. To avoid this, we explicitly retain references on the runs and write buffer blofs and only release them when the cursor is closed (see cursorRuns and cursorWBB below).

  • cursorRuns :: !(Vector (Ref (Run m h)))

    The runs held open by the cursor. We must release these references when the cursor gets closed.

  • cursorWBB :: !(Ref (WriteBufferBlobs m h))

    The write buffer blobs, which like the runs, we have to keep open untile the cursor is closed.

withCursor :: (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) => OffsetKey -> Table m h -> (Cursor m h -> m a) -> m a Source #

readCursor Source #

Arguments

:: forall m h res. (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) 
=> ResolveSerialisedValue 
-> Int

Maximum number of entries to read

-> Cursor m h 
-> (SerialisedKey -> SerialisedValue -> Maybe (WeakBlobRef m h) -> res)

How to map to a query result, different for normal/monoidal

-> m (Vector res) 

readCursorWhile Source #

Arguments

:: forall m h res. (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) 
=> ResolveSerialisedValue 
-> (SerialisedKey -> Bool)

Only read as long as this predicate holds

-> Int

Maximum number of entries to read

-> Cursor m h 
-> (SerialisedKey -> SerialisedValue -> Maybe (WeakBlobRef m h) -> res)

How to map to a query result, different for normal/monoidal

-> m (Vector res) 

readCursorWhile _ p n cursor _ reads elements until either:

  • n elements have been read already
  • p returns False for the key of an entry to be read
  • the cursor is drained

Consequently, once a call returned fewer than n elements, any subsequent calls with the same predicate p will return an empty vector.

Snapshots

data SnapshotLabel Source #

Custom, user-supplied text that is included in the metadata.

The main use case for a SnapshotLabel is for the user to supply textual information about the key/value/blob type for the table that corresponds to the snapshot. This information is used to dynamically check that a snapshot is opened at the correct key/value/blob type.

openSnapshot Source #

Arguments

:: (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) 
=> Session m h 
-> SnapshotLabel

Expected label

-> SnapshotTableType

Expected table type

-> TableConfigOverride

Optional config override

-> SnapshotName 
-> ResolveSerialisedValue 
-> m (Table m h) 

Multiple writable tables

duplicate :: (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) => Table m h -> m (Table m h) Source #

Table union

unions :: (MonadMask m, MonadMVar m, MonadST m, MonadSTM m) => NonEmpty (Table m h) -> m (Table m h) Source #

See unions.