Namespace: ContentStore
Interfaces
Type Aliases
DeleteEmbeddedContentArgs
Ƭ DeleteEmbeddedContentArgs: Object
Type declaration
| Name | Type | Description |
|---|---|---|
dataSources? | string[] | The names of the data sources for which to delete embedded content. |
inverseDataSources? | boolean | If true, delete pages that do NOT match the data sources in the query. |
page? | Page | The page for which to delete embedded content. |
Defined in
packages/mongodb-rag-core/src/contentStore/EmbeddedContent.ts:63
DeletePagesArgs
Ƭ DeletePagesArgs: Object
Type declaration
| Name | Type | Description |
|---|---|---|
dataSources? | string[] | The names of the sources to delete pages from. |
inverse? | boolean | If true, delete pages that do NOT match the query. |
permanent? | boolean | Permanently remove pages from the data store, rather than marking them as "deleted". |
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:90
EmbeddedContentStore
Ƭ EmbeddedContentStore: VectorStore<EmbeddedContent> & { close?: () => Promise<void> ; init?: () => Promise<void> ; metadata: { [k: string]: unknown; embeddingName: string } ; deleteEmbeddedContent: (args: DeleteEmbeddedContentArgs) => Promise<void> ; loadEmbeddedContent: (args: { page: Page }) => Promise<EmbeddedContent[]> ; updateEmbeddedContent: (args: { embeddedContent: EmbeddedContent[] ; page: Page }) => Promise<void> }
Data store of the embedded content.
Defined in
packages/mongodb-rag-core/src/contentStore/EmbeddedContent.ts:81
LoadPagesArgs
Ƭ LoadPagesArgs<QueryShape>: Object
Type parameters
| Name | Type |
|---|---|
QueryShape | unknown |
Type declaration
| Name | Type | Description |
|---|---|---|
query? | QueryShape | A custom query to refine the pages to load. |
sources? | string[] | The names of the sources to load pages from. If undefined, loads available pages from all sources. |
updated? | Date | If specified, refines the query to load pages with an updated date later or equal to the given date. |
urls? | string[] | If specified, refines the query to only load pages where the url is included in the list. |
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:65
MakeMongoDbEmbeddedContentStoreParams
Ƭ MakeMongoDbEmbeddedContentStoreParams: MakeMongoDbDatabaseConnectionParams & { collectionName?: string ; searchIndex: { embeddingName: string ; filters?: { path: string ; type: "filter" }[] ; name?: string ; numDimensions?: number } }
Defined in
packages/mongodb-rag-core/src/contentStore/MongoDbEmbeddedContentStore.ts:12
MakeMongoDbPageStoreParams
Ƭ MakeMongoDbPageStoreParams: MakeMongoDbDatabaseConnectionParams & { collectionName?: string }
Defined in
packages/mongodb-rag-core/src/contentStore/MongoDbPageStore.ts:29
MongoDbEmbeddedContentStore
Ƭ MongoDbEmbeddedContentStore: EmbeddedContentStore & DatabaseConnection & { metadata: { collectionName: string ; databaseName: string ; embeddingName: string ; embeddingPath: string } ; init: () => Promise<void> }
Defined in
packages/mongodb-rag-core/src/contentStore/MongoDbEmbeddedContentStore.ts:50
MongoDbPageStore
Ƭ MongoDbPageStore: DatabaseConnection & Omit<PageStore, "loadPages"> & { metadata: { collectionName: string ; databaseName: string } ; queryType: "mongodb" ; init: () => Promise<void> ; loadPages: (args?: LoadPagesArgs<Filter<PersistedPage>>) => Promise<PersistedPage[]> }
Defined in
packages/mongodb-rag-core/src/contentStore/MongoDbPageStore.ts:15
Page
Ƭ Page: Object
Represents a page from a data source.
Type declaration
| Name | Type | Description |
|---|---|---|
body | string | The text of the page. |
format | PageFormat | The file format of the page. This format determines how the page should be chunked and vector-embedded. |
metadata? | PageMetadata | Arbitrary metadata for page. |
sourceName | string | Data source name. |
title? | string | A human-readable title. |
url | string | - |
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:6
PageAction
Ƭ PageAction: "created" | "updated" | "deleted"
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:48
PageFormat
Ƭ PageFormat: typeof pageFormats[number]
A canonical page format.
Defined in
packages/mongodb-rag-core/src/contentStore/PageFormat.ts:78
PageMetadata
Ƭ PageMetadata: Object
Index signature
▪ [k: string]: unknown
Type declaration
| Name | Type | Description |
|---|---|---|
page? | Record<string, unknown> | Page-level metadata. Should not be chunked. |
tags? | string[] | Arbitrary tags. |
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:36
PageStore
Ƭ PageStore: Object
Data store for Page objects.
Type declaration
| Name | Type | Description |
|---|---|---|
close? | () => Promise<void> | Close connection to data store. |
init? | () => Promise<void> | Initialize the store. |
metadata? | { [k: string]: unknown; } | Additional implementation-specific metadata about the store. This metadata is not directly used by the store itself, but may be useful for testing, debugging, and logging. |
queryType? | "mongodb" | string | The format that the store uses for custom queries. If not specified, the store does not allow custom queries. |
deletePages | (args?: DeletePagesArgs) => Promise<void> | Deletes pages from the store. |
loadPages | (args?: LoadPagesArgs<unknown>) => Promise<PersistedPage[]> | Loads pages from the Page store. |
updatePages | (pages: PersistedPage[]) => Promise<void> | Updates or adds the given pages in the store. |
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:109
PersistedPage
Ƭ PersistedPage: Page & { action: PageAction ; updated: Date }
Represents a Page stored in the database.
Defined in
packages/mongodb-rag-core/src/contentStore/Page.ts:53
Variables
pageFormats
• Const pageFormats: ("txt" | "md" | "mdx" | "restructuredtext" | "csv" | "json" | "yaml" | "toml" | "xml" | "openapi-yaml" | "openapi-json" | "graphql" | "c" | "cpp" | "csharp" | "go" | "html" | "java" | "javascript" | "kotlin" | "latex" | "objective-c" | "php" | "python" | "ruby" | "rust" | "scala" | "shell" | "swift" | "typescript")[]
The list of canonical file formats that we support for pages.
Defined in
packages/mongodb-rag-core/src/contentStore/PageFormat.ts:49
Functions
asPageFormat
▸ asPageFormat(str): undefined | "txt" | "md" | "mdx" | "restructuredtext" | "csv" | "json" | "yaml" | "toml" | "xml" | "openapi-yaml" | "openapi-json" | "graphql" | "c" | "cpp" | "csharp" | "go" | "html" | "java" | "javascript" | "kotlin" | "latex" | "objective-c" | "php" | "python" | "ruby" | "rust" | "scala" | "shell" | "swift" | "typescript"
Maps a string to the canonical page format it represents.
Parameters
| Name | Type |
|---|---|
str | string |
Returns
undefined | "txt" | "md" | "mdx" | "restructuredtext" | "csv" | "json" | "yaml" | "toml" | "xml" | "openapi-yaml" | "openapi-json" | "graphql" | "c" | "cpp" | "csharp" | "go" | "html" | "java" | "javascript" | "kotlin" | "latex" | "objective-c" | "php" | "python" | "ruby" | "rust" | "scala" | "shell" | "swift" | "typescript"
The canonical page format, or undefined if the string is not a recognized page format.
Defined in
packages/mongodb-rag-core/src/contentStore/PageFormat.ts:67
getChangedPages
▸ getChangedPages(«destructured»): Promise<{ created: PersistedPage[] ; deleted: PersistedPage[] ; updated: PersistedPage[] }>
Given sets of old and new pages, returns the pages that need to be created, updated, or deleted.
Parameters
| Name | Type |
|---|---|
«destructured» | Object |
› newPages | Page[] |
› oldPages | Omit<PersistedPage, "updated">[] |
› sourceName? | string |
Returns
Promise<{ created: PersistedPage[] ; deleted: PersistedPage[] ; updated: PersistedPage[] }>
Defined in
packages/mongodb-rag-core/src/contentStore/getChangedPages.ts:9
isPageFormat
▸ isPageFormat(str): str is "txt" | "md" | "mdx" | "restructuredtext" | "csv" | "json" | "yaml" | "toml" | "xml" | "openapi-yaml" | "openapi-json" | "graphql" | "c" | "cpp" | "csharp" | "go" | "html" | "java" | "javascript" | "kotlin" | "latex" | "objective-c" | "php" | "python" | "ruby" | "rust" | "scala" | "shell" | "swift" | "typescript"
Type guard to check if a string is a canonical page format.
Parameters
| Name | Type |
|---|---|
str | string |
Returns
str is "txt" | "md" | "mdx" | "restructuredtext" | "csv" | "json" | "yaml" | "toml" | "xml" | "openapi-yaml" | "openapi-json" | "graphql" | "c" | "cpp" | "csharp" | "go" | "html" | "java" | "javascript" | "kotlin" | "latex" | "objective-c" | "php" | "python" | "ruby" | "rust" | "scala" | "shell" | "swift" | "typescript"
Defined in
packages/mongodb-rag-core/src/contentStore/PageFormat.ts:83
makeMongoDbEmbeddedContentStore
▸ makeMongoDbEmbeddedContentStore(«destructured»): MongoDbEmbeddedContentStore
Parameters
| Name | Type |
|---|---|
«destructured» | MakeMongoDbEmbeddedContentStoreParams |
Returns
Defined in
packages/mongodb-rag-core/src/contentStore/MongoDbEmbeddedContentStore.ts:61
makeMongoDbPageStore
▸ makeMongoDbPageStore(«destructured»): MongoDbPageStore
Data store for Page objects using MongoDB.
Parameters
| Name | Type |
|---|---|
«destructured» | MakeMongoDbPageStoreParams |
Returns
Defined in
packages/mongodb-rag-core/src/contentStore/MongoDbPageStore.ts:40
pageFormat
▸ pageFormat(str, defaultPageFormat?): PageFormat
Converts a string to a canonical page format. If the string is not a recognized page format or a synonym for one, this returns a default page format.
Parameters
| Name | Type | Default value |
|---|---|---|
str | string | undefined |
defaultPageFormat | "txt" | "md" | "mdx" | "restructuredtext" | "csv" | "json" | "yaml" | "toml" | "xml" | "openapi-yaml" | "openapi-json" | "graphql" | "c" | "cpp" | "csharp" | "go" | "html" | "java" | "javascript" | "kotlin" | "latex" | "objective-c" | "php" | "python" | "ruby" | "rust" | "scala" | "shell" | "swift" | "typescript" | "txt" |
Returns
Defined in
packages/mongodb-rag-core/src/contentStore/PageFormat.ts:92
pageIdentity
▸ pageIdentity(«destructured»): Object
Returns a query filter that represents a unique page in the system.
Parameters
| Name | Type |
|---|---|
«destructured» | Page |
Returns
Object
| Name | Type |
|---|---|
sourceName | string |
url | string |
Defined in
packages/mongodb-rag-core/src/contentStore/pageIdentity.ts:6
persistPages
▸ persistPages(«destructured»): Promise<void>
Persists pages that have changed.
Parameters
| Name | Type |
|---|---|
«destructured» | Object |
› pages | Page[] |
› sourceName | string |
› store | PageStore |
Returns
Promise<void>
Defined in
packages/mongodb-rag-core/src/contentStore/updatePages.ts:51
updateEmbeddedContent
▸ updateEmbeddedContent(«destructured»): Promise<void>
(Re-)embeddedContent the pages in the page store that have changed since the given date and stores the embeddedContent in the embeddedContent store.
Parameters
| Name | Type |
|---|---|
«destructured» | Object |
› chunkOptions? | Partial<ChunkOptions> |
› concurrencyOptions? | EmbedConcurrencyOptions |
› embeddedContentStore | EmbeddedContentStore |
› embedder | Embedder |
› pageStore | PageStore |
› since | Date |
› sourceNames? | string[] |
Returns
Promise<void>
Defined in
packages/mongodb-rag-core/src/contentStore/updateEmbeddedContent.ts:25
updateEmbeddedContentForPage
▸ updateEmbeddedContentForPage(«destructured»): Promise<void>
Parameters
| Name | Type |
|---|---|
«destructured» | Object |
› chunkOptions? | Partial<ChunkOptions> |
› concurrencyOptions? | EmbedConcurrencyOptions |
› embedder | Embedder |
› page | PersistedPage |
› store | EmbeddedContentStore |
Returns
Promise<void>
Defined in
packages/mongodb-rag-core/src/contentStore/updateEmbeddedContent.ts:91
updatePages
▸ updatePages(«destructured»): Promise<void>
Fetches pages from data sources and stores those that have changed in the data store.
Parameters
| Name | Type |
|---|---|
«destructured» | Object |
› concurrencyOptions? | PageConcurrencyOptions |
› pageStore | PageStore |
› sources | DataSource[] |
Returns
Promise<void>
Defined in
packages/mongodb-rag-core/src/contentStore/updatePages.ts:18