Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
config | MarkdownLoaderConfig | Required | Configuration object for Markdown loading behavior |
Functions
__init__
Initializes the MarkdownLoader with its specific configuration.
Parameters:
config(MarkdownLoaderConfig): Configuration object for Markdown loading behavior
get_supported_extensions
Gets the list of supported file extensions.
Returns:
List[str]: List of supported file extensions (.md,.markdown)
load
Loads documents from the given Markdown source(s) synchronously.
Parameters:
source(Union[str, Path, List[Union[str, Path]]]): Markdown source(s) to load from
List[Document]: List of loaded documents
aload
Loads documents from the given Markdown source(s) asynchronously.
Parameters:
source(Union[str, Path, List[Union[str, Path]]]): Markdown source(s) to load from
List[Document]: List of loaded documents
batch
A simple synchronous batch load implementation.
Parameters:
sources(List[Union[str, Path]]): List of Markdown sources to load
List[Document]: List of loaded documents
abatch
An efficient asynchronous batch load implementation.
Parameters:
sources(List[Union[str, Path]]): List of Markdown sources to load
List[Document]: List of loaded documents
_load_single_file
Loads, parses, and chunks a single Markdown file based on the config.
Parameters:
file_path(Path): Path to the Markdown file
List[Document]: List of documents created from the Markdown file
_chunk_tokens
Splits a list of tokens into chunks if split_by_heading is configured.
Parameters:
tokens(List[Token]): List of markdown tokens to chunk
List[List[Token]]: List of token chunks
_process_chunk
Processes a single chunk of tokens into a Document, extracting content and metadata.
Parameters:
tokens(List[Token]): List of tokens to processdocument_id(str): Document IDmetadata(Dict[str, Any]): Metadata dictionary
Optional[Document]: Created document or None if empty