Skip to main content

DirectoryLoader

Compatibility

Only available on Node.js.

This notebook provides a quick overview for getting started with DirectoryLoader document loaders. For detailed documentation of all DirectoryLoader features and configurations head to the API reference.

This example goes over how to load data from folders with multiple files. The second argument is a map of file extensions to loader factories. Each file will be passed to the matching loader, and the resulting documents will be concatenated together.

Example folder:

src/document_loaders/example_data/example/
├── example.json
├── example.jsonl
├── example.txt
└── example.csv

Overview

Integration details

ClassPackageCompatibilityLocalPY support
DirectoryLoaderlangchainNode-only

Setup

To access DirectoryLoader document loader you’ll need to install the langchain package.

Installation

The LangChain DirectoryLoader integration lives in the langchain package:

yarn add langchain @langchain/core

Instantiation

Now we can instantiate our model object and load documents:

import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
import {
JSONLoader,
JSONLinesLoader,
} from "langchain/document_loaders/fs/json";
import { TextLoader } from "langchain/document_loaders/fs/text";
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";

const loader = new DirectoryLoader(
"../../../../../../examples/src/document_loaders/example_data",
{
".json": (path) => new JSONLoader(path, "/texts"),
".jsonl": (path) => new JSONLinesLoader(path, "/html"),
".txt": (path) => new TextLoader(path),
".csv": (path) => new CSVLoader(path, "text"),
}
);

Load

const docs = await loader.load();
// disable console.warn calls
console.warn = () => {};
docs[0];
Document {
pageContent: 'Foo\nBar\nBaz\n\n',
metadata: {
source: '/Users/bracesproul/code/lang-chain-ai/langchainjs/examples/src/document_loaders/example_data/example.txt'
},
id: undefined
}
console.log(docs[0].metadata);
{
source: '/Users/bracesproul/code/lang-chain-ai/langchainjs/examples/src/document_loaders/example_data/example.txt'
}

API reference

For detailed documentation of all DirectoryLoader features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain.document_loaders_fs_directory.DirectoryLoader.html


Was this page helpful?


You can also leave detailed feedback on GitHub.