BentoML code evolution #5325
Unanswered
andrehora
asked this question in
Show and tell
Replies: 2 comments
|
BentoML multi-service deployments are powerful but require careful architecture. At RevolutionAI (https://revolutionai.io), we run multi-service bentos for our AI pipelines. Patterns that work:
@bentoml.service
class OrchestratorService:
embedding = bentoml.depends(EmbeddingService)
llm = bentoml.depends(LLMService)
@bentoml.api
async def process(self, text: str):
emb = await self.embedding.embed(text)
return await self.llm.generate(emb)
What specific multi-service pattern are you trying to implement? |
0 replies
|
BentoML code evolution has been impressive! At RevolutionAI (https://revolutionai.io) we migrated from 0.x to 1.x. Key improvements:
Migration tips: # Old
@bentoml.env(pip_dependencies=[...])
class Service:
pass
# New
@bentoml.service(resources={"cpu": "2"})
class Service:
passWorth the upgrade! |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hello guys,
Just to share some code evolution metrics 📊 of BentoML in the last five years:
https://andrehora.github.io/gitevo-examples/python/bentoml.html
Try in your own Python repository:
For example:
More info: https://github.com/andrehora/gitevo
All reactions