Add Trellis as a private LLM provider

[ocramz]

Feature request

Currently this package assumes that the user workstation has a GPU or enough local capacity to run a LLM.
Trellis splits an LLM across multiple workstations with an encrypted channel between them, which enables large models and multiple model consumers.

Once an LLM is loaded onto a grid, the CLI exposes an Anthropic-like API which can be consumed by clients such as Nextcloud/llm2 : https://trellis.unfoldml.com/docs#api

Disclaimer 1: Trellis is a paid service with a monthly plan, and the client (where the compute runs) is currently proprietary software.
Disclaimer 2: I am the founder/author of Trellis, actively developing the project

Thank you and looking forward to your feedback !

[edward-ly]

Thanks for your suggestion, but we don't think it makes sense to integrate this within the llm2 app, since it is supposed to be a fully local LLM provider (without any additional networking). You could instead write a separate PHP app that implements a provider that connects to any service that is compatible with the Anthropic API (with Trellis being just one possible example). It could be called something like "Anthropic integration app", similar to the OpenAI integration app. While the Anthropic API integration is not in our roadmap at this time, you are free to contribute by writing this app yourself if you're up for it!