feat(fastembed): allow passing model_kwargs to dense embedders#3558
Open
RahilOp wants to merge 1 commit into
Open
feat(fastembed): allow passing model_kwargs to dense embedders#3558RahilOp wants to merge 1 commit into
RahilOp wants to merge 1 commit into
Conversation
FastembedTextEmbedder and FastembedDocumentEmbedder had no way to forward extra arguments to the underlying fastembed TextEmbedding model, so users could not select an ONNX execution provider (e.g. CUDAExecutionProvider) to run embeddings on GPU. Add a model_kwargs parameter to both embedders and the dense backend, mirroring the existing model_kwargs support in the sparse embedders. The kwargs are forwarded to TextEmbedding, enabling GPU providers and other model options.
Contributor
|
Heads-up for maintainers This PR is from a fork and touches integrations whose integration tests require API keys. Affected integrations:
Please run the integration tests locally ( |
Contributor
Coverage report (fastembed)Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
Proposed Changes:
FastembedTextEmbedderandFastembedDocumentEmbedderdid not expose a way to forward extra arguments to the underlying fastembedTextEmbeddingmodel. As a result, users could not select an ONNX execution provider (e.g.CUDAExecutionProvider) to run embeddings on GPU, which is the standard way to enable GPU support in fastembed.This adds a
model_kwargsparameter to both dense embedders and the dense embedding backend, forwarding the kwargs toTextEmbedding(...). This mirrors the existingmodel_kwargssupport already present in the sparse embedders (FastembedSparseTextEmbedder/FastembedSparseDocumentEmbedder), so the design and serialization are consistent across the integration.Usage:
Note: running on GPU also requires installing
fastembed-gpuin the environment (as per the fastembed docs); this PR only exposes the passthrough so the option becomes usable, without changing the integration's dependencies.How did you test it?
Unit tests (
hatch run test:unit), following the existing sparse-embedder test patterns:model_kwargstoTextEmbeddingmodel_kwargsis stored on both embedders and round-trips throughto_dictwarm_uppassesmodel_kwargsto the backend factoryAI tooling assisted with parts of this change.
Notes for the reviewer
The dense backend cache key now also includes
local_files_onlyandmodel_kwargs, matching the sparse backend's key so backends differing only in those args aren't incorrectly shared.Checklist
feat: