Included PR 12 + Poetry + OpenAI fixes#14
Conversation
|
cc @krrishdholakia - since this is a merge with your PR |
There was a problem hiding this comment.
what is the purpose of adding poetry here ?
If it's for deploying new versions of the package i'm currently working on a github action to deploy from Github -> PyPI
There was a problem hiding this comment.
You have a great point here 🤔
Mainly for the convenience of development, but it'll primarily be used as a package. There's also a build functionality to package into .whl, but again - PyPI is preferable.
And if doing a GH action, it may well render that unneeded 🥲
P.S. The action option seems awesome!
There was a problem hiding this comment.
Handling PyPI packages using https://github.com/BerriAI/litellm/actions/workflows/publish_pypi.yml
There was a problem hiding this comment.
@zakhar-kogan do you need poetry for your local development ?
There was a problem hiding this comment.
@zakhar-kogan do you need poetry for your local development ?
Not necessarily; I can manage without it + at least add pyproject.toml and poetry.lock to .gitignore. Should I do it?
There was a problem hiding this comment.
For tracking can we add poetry and pyproject in a new PR ?
There was a problem hiding this comment.
Sure, will do that in a separate commit/PR
There was a problem hiding this comment.
lets do this in another PR, and no need for a gitignore
There was a problem hiding this comment.
@krrishdholakia from PR #12 - I don't think we should default all users with a 10s timeout
There was a problem hiding this comment.
Maybe we could delete the default 300 token limit in PR #12? 🤔
I.e. make behavior in check with default: unlimited by default, optional limit.
PS. I'll also add the temperature setting in the next PR as I'll require it at work 😅
There was a problem hiding this comment.
from @krrishdholakia: Reason for token limit - Anthropic requires token limits. Their defaults are 300.
There was a problem hiding this comment.
Order for PRs pushing to main:
- @krrishdholakia pushing latest version with litellm client
- @zakhar-kogan adding poetry
- @zakhar-kogan this current PR Included PR 12 + Poetry + OpenAI fixes #14
There was a problem hiding this comment.
@krrishdholakia set timeout for 60s
|
…rite bypass
Exercise every state-mutating endpoint as PROXY_ADMIN_VIEW_ONLY and
confirm they all return 403:
* POST /v2/agents
* PATCH /v2/agents/{id}
* DELETE /v2/agents/{id}
* POST /v2/sessions
* DELETE /v2/sessions/{id}
* POST /v2/sessions/{id}/runs
* POST /v2/sessions/{id}/runs/{rid}/cancel
* POST /v2/sessions/{id}/followup
Plus a happy-path test confirming view-only admins still get
cross-tenant READ access (the whole point of the role).
Greptile P1 SECURITY regression coverage (validation #14).
2 participants
Needs to be tested.
max_tokens_to_sample⇢ max_tokens. The first one is used in Anthropic as I get it.