Skip to content

feat(fastembed): allow passing model_kwargs to dense embedders#3558

Open
RahilOp wants to merge 1 commit into
deepset-ai:mainfrom
RahilOp:feat/fastembed-dense-model-kwargs
Open

feat(fastembed): allow passing model_kwargs to dense embedders#3558
RahilOp wants to merge 1 commit into
deepset-ai:mainfrom
RahilOp:feat/fastembed-dense-model-kwargs

Conversation

@RahilOp

@RahilOp RahilOp commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

FastembedTextEmbedder and FastembedDocumentEmbedder did not expose a way to forward extra arguments to the underlying fastembed TextEmbedding model. As a result, users could not select an ONNX execution provider (e.g. CUDAExecutionProvider) to run embeddings on GPU, which is the standard way to enable GPU support in fastembed.

This adds a model_kwargs parameter to both dense embedders and the dense embedding backend, forwarding the kwargs to TextEmbedding(...). This mirrors the existing model_kwargs support already present in the sparse embedders (FastembedSparseTextEmbedder / FastembedSparseDocumentEmbedder), so the design and serialization are consistent across the integration.

Usage:

embedder = FastembedTextEmbedder(
    model="BAAI/bge-small-en-v1.5",
    model_kwargs={"providers": ["CUDAExecutionProvider"]},
)

Note: running on GPU also requires installing fastembed-gpu in the environment (as per the fastembed docs); this PR only exposes the passthrough so the option becomes usable, without changing the integration's dependencies.

How did you test it?

Unit tests (hatch run test:unit), following the existing sparse-embedder test patterns:

  • backend forwards model_kwargs to TextEmbedding
  • model_kwargs is stored on both embedders and round-trips through to_dict
  • warm_up passes model_kwargs to the backend factory

AI tooling assisted with parts of this change.

Notes for the reviewer

The dense backend cache key now also includes local_files_only and model_kwargs, matching the sparse backend's key so backends differing only in those args aren't incorrectly shared.

Checklist

FastembedTextEmbedder and FastembedDocumentEmbedder had no way to forward
extra arguments to the underlying fastembed TextEmbedding model, so users
could not select an ONNX execution provider (e.g. CUDAExecutionProvider)
to run embeddings on GPU.

Add a model_kwargs parameter to both embedders and the dense backend,
mirroring the existing model_kwargs support in the sparse embedders. The
kwargs are forwarded to TextEmbedding, enabling GPU providers and other
model options.
@RahilOp RahilOp requested a review from a team as a code owner July 7, 2026 10:24
@RahilOp RahilOp requested review from julian-risch and removed request for a team July 7, 2026 10:24
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Heads-up for maintainers

This PR is from a fork and touches integrations whose integration tests require API keys.
Those tests are skipped in CI because fork PRs don't have access to repo secrets for security reasons.

Affected integrations:

  • fastembed

Please run the integration tests locally (hatch run test:integration inside each folder) before approving.

@github-actions github-actions Bot added the type:documentation Improvements or additions to documentation label Jul 7, 2026
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Coverage report (fastembed)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/fastembed/src/haystack_integrations/components/embedders/fastembed
  fastembed_document_embedder.py
  fastembed_text_embedder.py
  integrations/fastembed/src/haystack_integrations/components/embedders/fastembed/embedding_backend
  fastembed_backend.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:fastembed type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow passing provider option to FastEmbed to allow embedding on the GPU

1 participant