GridFS

GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16MB.

Instead of storing a file in a single document, GridFS divides a file into parts, or chunks, and stores each of those chunks as a separate document. By default GridFS limits chunk size to 255kb. GridFS uses two collections to store files. The chunks collection stores the file chunks, and the files collection stores the file metadata.

When you query a GridFS store for a file, the driver or client will reassemble the chunks as needed. GridFS is useful not only for storing files that exceed 16MB but also for storing any files for which you want access without having to load the entire file into memory.

The following code snippets come from the GridFSTour.java example code that can be found with the driver source.

Async Streams

As there are multiple API’s for Asynchronous I/O on the JVM the GridFS library uses a flexible interfaces for asynchronous input and output. The AsyncInputStream interface represents an InputStream and the AsyncOutputStream interface represents an OutputStream.

In addition to these interfaces there are the following helpers:

• AsyncStreamHelper which provides support for:

  • byte[]
  • ByteBuffer
  • InputStream - note: input streams are blocking
  • OutputStream - note: output streams are blocking

• AsynchronousChannelHelper which provides support for:

  • AsynchronousByteChannel
  • AsynchronousFileChannel

These interfaces should be easy to wrap for any alternative asynchronous I/O implementations such as Akka, Netty or Vertx.

Connecting to GridFS

Interactions with GridFS are done via the GridFSBucket class.

Creating a GridFSBucket requires an instance of a MongoDatabase and you can optionally provide a custom bucket name.

The following example shows how to create a GridFSBucket:

// Create a gridFSBucket using the default bucket name "fs"
val gridFSBucket: GridFSBucket = GridFSBucket(myDatabase)

// Create a gridFSBucket with a custom bucket name "files"
val customGridFSBucket: GridFSBucket = GridFSBuckets(myDatabase, "files")

Uploading to GridFS

There are two main ways to upload data into GridFS.

UploadFromStream

The uploadFromStream method reads the contents of an AsyncInputStream and saves it to the GridFSBucket. The size of the chunks defaults to 255 kb, but can be configured via the GridFSUploadOptions.

The following example uploads an AsyncInputStream into GridFSBucket:

// Get the input stream
val inputPath: Path = Paths.get("/tmp/mongodb-tutorial.pdf")
val streamToDownloadTo: AsynchronousFileChannel = AsynchronousFileChannel.open(inputPath, StandardOpenOptionRead)
val streamToUploadFrom: AsyncInputStream = channelToInputStream(streamToDownloadTo) // Using the AsynchronousChannelHelper

// Create some custom options
val options: GridFSUploadOptions = new GridFSUploadOptions().chunkSizeBytes(1024 * 1204).metadata(Document("type" -> "presentation"))

val fileId: ObjectId = gridFSBucket.uploadFromStream("mongodb-tutorial", streamToUploadFrom, options).headResult()
streamToUploadFrom.close()

OpenUploadStream

The openUploadStream method returns a GridFSUploadStream which extends AsyncOutputStream and can be written to.

The GridFSUploadStream buffers data until it reaches the chunkSizeBytes and then inserts the chunk into the chunks collection. When the GridFSUploadStream is closed, the final chunk is written and the file metadata is inserted into the files collection.

The following example uploads an into GridFSBucket via the returned OutputStream:

val data = ByteBuffer.wrap("Data to upload into GridFS".getBytes(StandardCharsets.UTF_8))
val uploadStream: GridFSUploadStream = gridFSBucket.openUploadStream("sampleData")
uploadStream.write(data).headResult()
uploadStream.close().headResult()

Finding files stored in GridFS

To find the files stored in the GridFSBucket use the find method.

The following example prints out the filename of each file stored:

gridFSBucket.find().results().foreach(file => println(s" - ${file.getFilename}"))

You can also provide a custom filter to limit the results returned. The following example prints out the filenames of all files with a “image/png” value set as the contentType in the user defined metadata document:

gridFSBucket.find(Filters.equal("metadata.contentType", "image/png")).results().foreach(file => println(s" > ${file.getFilename}"))

Downloading from GridFS

There are four main ways to download data from GridFS.

DownloadToStream

The downloadToStream method reads the contents from MongoDB and writes the data directly to the provided AsyncOutputStream.

The following example downloads a file into the provided OutputStream:

val outputPath: Path = Paths.get("/tmp/mongodb-tutorial.txt")
var streamToDownloadTo: AsynchronousFileChannel = AsynchronousFileChannel.open(outputPath, StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE, StandardOpenOption.DELETE_ON_CLOSE)
gridFSBucket.downloadToStream(fileId, channelToOutputStream(streamToDownloadTo)).headResult()
streamToDownloadTo.close()

DownloadToStreamByName

If you don’t know the ObjectId of the file you want to download, then you use the downloadToStreamByName method. By default it will download the latest version of the file. Use the GridFSDownloadByNameOptions to configure which version to download.

The following example downloads the original version of the file named “mongodb-tutorial” into the OutputStream:

streamToDownloadTo = AsynchronousFileChannel.open(outputPath, StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE, StandardOpenOption.DELETE_ON_CLOSE)
val downloadOptions: GridFSDownloadOptions = new GridFSDownloadOptions().revision(0)
gridFSBucket.downloadToStream("mongodb-tutorial", channelToOutputStream(streamToDownloadTo), downloadOptions).headResult()
streamToDownloadTo.close().headResult()

OpenDownloadStream

The openDownloadStream method returns a GridFSDownloadStream which extends AsyncInputStream and can be read from.

The following example reads from the GridFSBucket via the returned AsyncInputStream:

val dstByteBuffer: ByteBuffer = ByteBuffer.allocate(1024 * 1024)
val downloadStream: GridFSDownloadStream = gridFSBucket.openDownloadStream(fileId)
downloadStream.read(dstByteBuffer).map(result => {
  dstByteBuffer.flip
  val bytes: Array[Byte] = new Array[Byte](result)
  dstByteBuffer.get(bytes)
  println(new String(bytes, StandardCharsets.UTF_8))
}).headResult()

OpenDownloadStream by name

You can also open a GridFSDownloadStream by searching against the filename, using the openDownloadStream method. By default it will download the latest version of the file. Use the GridFSDownloadByNameOptions to configure which version to download.

The following example downloads the latest version of the file named “sampleData” into the dstByteBuffer ByteBuffer:

val downloadStreamByName: GridFSDownloadStream = gridFSBucket.openDownloadStream("sampleData")
downloadStreamByName.read(dstByteBuffer).map(result => {
  dstByteBuffer.flip
  val bytes: Array[Byte] = new Array[Byte](result)
  dstByteBuffer.get(bytes)
  println(new String(bytes, StandardCharsets.UTF_8))
}).headResult()

Renaming files

If you should need to rename a file, then the rename method can be used.

The following example renames a file to “mongodbTutorial”:

gridFSBucket.rename(fileId, "mongodbTutorial").results()

Deleting files

To delete a file from the GridFSBucket use the delete method.

The following example deletes a file from the GridFSBucket:

gridFSBucket.delete(fileId).results()