module System.FS.BlockIO ( -- * Description -- $description -- * Re-exports module System.FS.BlockIO.API , module System.FS.BlockIO.IO -- * Example -- $example ) where import System.FS.BlockIO.API import System.FS.BlockIO.IO {------------------------------------------------------------------------------- Examples -------------------------------------------------------------------------------} {- $description The 'HasBlockIO' record type defines an /abstract interface/. A value of a 'HasBlockIO' type is what we call an /instance/ of the abstract interface, and an instance is produced by a function that we call an /implementation/. In principle, we can have multiple instances of the same implementation. There are currently two known implementations of the interface: * An implementation using the real file system, which can be found in the "System.FS.BlockIO.IO" module. This implementation is platform-dependent, but has largely similar observable behaviour. * An implementation using a simulated file system, which can be found in the @System.FS.BlockIO.Sim@ module of the @blockio:sim@ sublibrary. This implementation is uniform across platforms. The 'HasBlockIO' abstract interface is an extension of the 'HasFS' abstract interface that is provided by the [@fs-api@](https://hackage.haskell.org/package/fs-api) package. Whereas 'HasFS' defines many primitive functions, for example for opening a file, the main feature of 'HasBlockIO' is to define a function for performing batched I\/O. As such, users of @blockio@ will more often than not need to pass both a 'HasFS' and a 'HasBlockIO' instance to their functions. -} {- $example >>> import Control.Monad >>> import Control.Monad.Primitive >>> import Data.Primitive.ByteArray >>> import Data.Vector qualified as V >>> import Data.Word >>> import Debug.Trace >>> import System.FS.API as FS >>> import System.FS.BlockIO.IO >>> import System.FS.BlockIO.API >>> import System.FS.IO The main feature of the 'HasBlockIO' abstract interface is that it provides a function for performing batched I\/O using 'submitIO'. Depending on the implementation of the interface, performing I\/O in batches concurrently using 'submitIO' can be much faster than performing each I\/O operation in a sequential order. We will not go into detail about this performance consideration here, but more information can be found in the "System.FS.BlockIO.IO" module. Instead, we will show an example of how 'submitIO' can be used in your own projects. We aim to build an example that writes some contents to a file using 'submitIO', and then reads the contents out again using 'submitIO'. The file contents will simply be bytes. >>> type Byte = Word8 The first part of the example is to write out bytes to a given file path using 'submitIO'. We define a @writeFile@ function that does just that. The file is assumed to not exist already. The bytes, which are provided as separate bytes, are written into a buffer (a mutable byte array). Note that the buffer should be /pinned/ memory to prevent pointer aliasing. In the case of write operations, this buffer is used to communicate to the backend what the bytes are that should be written to disk. For simplicity, we create a separate 'IOOpWrite' instruction for each byte. This instruction requires information about the write operation. In order of appearence these are: the file handle to write bytes to, the offset into that file, the buffer, the offset into that buffer, and the number of bytes to write. Finally, all instructions are batched together and submitted in one go using 'submitIO'. For each instruction, an 'IOResult' is returned, which describes in this case the number of written bytes. If any of the instructions failed to be performed, an error is thrown. We print the 'IOResult's to standard output. Note that in real scenarios it would be much more performant to aggregate the bytes into larger chunks, and to create an instruction for each of those chunks. A sensible size for those chunks would be the disk page size (4Kb for example), or a multiple of that disk page size. The disk page size is typically the smallest chunk of memory that can be written to or read from the disk. In some cases it is also desirable or even required that the buffers are aligned to the disk page size. For example, alignment is required when using direct I\/O. >>> :{ writeFile :: HasFS IO HandleIO -> HasBlockIO IO HandleIO -> FsPath -> [Byte] -> IO () writeFile hasFS hasBlockIO file bytes = do let numBytes = length bytes FS.withFile hasFS file (FS.WriteMode FS.MustBeNew) $ \h -> do buffer <- newPinnedByteArray numBytes forM_ (zip [0..] bytes) $ \(i, byte) -> let bufferOffset = fromIntegral i in writeByteArray @Byte buffer bufferOffset byte results <- submitIO hasBlockIO $ V.fromList [ IOOpWrite h fileOffset buffer bufferOffset 1 | i <- take numBytes [0..] , let fileOffset = fromIntegral i bufferOffset = FS.BufferOffset i ] print results :} The second part of the example is to read a given number of bytes from a given file path using 'submitIO'. We define a @readFile@ function that follows the same general structure and behaviour as @writeFile@, but @readFile@ is of course reading bytes instead of writing bytes. >>> :{ readFile :: HasFS IO HandleIO -> HasBlockIO IO HandleIO -> FsPath -> Int -> IO [Byte] readFile hasFS hasBlockIO file numBytes = do FS.withFile hasFS file FS.ReadMode $ \h -> do buffer <- newPinnedByteArray numBytes results <- submitIO hasBlockIO $ V.fromList [ IOOpRead h fileOffset buffer bufferOffset numBytes | i <- [0..3] , let fileOffset = fromIntegral i bufferOffset = FS.BufferOffset i numBytes = 1 ] print results forM (take numBytes [0..]) $ \i -> let bufferOffset = i in readByteArray @Byte buffer i :} Now we can combine @writeFile@ and @readFile@ into a very small example called @writeReadFile@, which does what we set out to do: write a few bytes to a (temporary) file and read them out again using 'submitIO'. We also print the bytes that were written and the bytes that were read, so that the user can check by hand whether the bytes match. >>> :{ writeReadFile :: HasFS IO HandleIO -> HasBlockIO IO HandleIO -> IO () writeReadFile hasFS hasBlockIO = do let file = FS.mkFsPath ["simple_example.txt"] let bytesIn = [1,2,3,4] print bytesIn writeFile hasFS hasBlockIO file bytesIn bytesOut <- readFile hasFS hasBlockIO file 4 print bytesOut FS.removeFile hasFS file :} In order to run @writeReadFile@, we will need 'HasFS' and 'HasBlockIO' instances. This is where the separation between interface and implementation shines: @writeReadFile@ is agnostic to the implementations of the the abstract interfaces, so we could pick any implementations and slot them in. For this example we will use the /real/ implementation from "System.FS.BlockIO.IO", but we could have used the /simulated/ implementation from the @blockio:sim@ sub-library just as well. We define the @example@ function, which uses 'withIOHasBlockIO' to instantiate both a 'HasFS' and 'HasBlockIO' instance, which we pass to 'writeReadFile'. >>> :{ example :: IO () example = withIOHasBlockIO (MountPoint "") defaultIOCtxParams $ \hasFS hasBlockIO -> writeReadFile hasFS hasBlockIO :} Finally, we can run the example to produce some output. As we can see, the input bytes match the output bytes. >>> example [1,2,3,4] [IOResult 1,IOResult 1,IOResult 1,IOResult 1] [IOResult 1,IOResult 1,IOResult 1,IOResult 1] [1,2,3,4] -}