You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process.
bentoml.exceptions.BentoMLException: Failed to load bento model because it does not contain a 'model.yaml'
To reproduce
1-add the model like they describe here [https://github.com/bentoml/OpenLLM/blob/main/ADDING_NEW_MODEL.md]
2-pip install openllm
3-openllm -h
4-Run the command to add the TRUST_REMOTE_CODE=True openllm start CohereForAI/aya-101 model.
Logs
It is recommended to specify the backend explicitly. Cascading backend might lead to unexpected behaviour.
Traceback (most recent call last):
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
return io.open(
FileNotFoundError: [Errno 2] No such file or directory: b'/home/ubuntu/bentoml/models/vllm-cohereforai--aya-101/latest'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 144, in get
_tag.version = self._fs.readtext(_tag.latest_path())
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/base.py", line 693, in readtext
self.open(
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 643, in open
with convert_os_errors("open", path):
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/error_tools.py", line 89, in __exit__
reraise(fserror, fserror(self._path, exc=exc_value), traceback)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/six.py", line 718, in reraise
raise value.with_traceback(tb)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
return io.open(
fs.errors.ResourceNotFound: resource 'vllm-cohereforai--aya-101/latest' not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
return io.open(
FileNotFoundError: [Errno 2] No such file or directory: b'/home/ubuntu/bentoml/models/vllm-cohereforai--aya-101/7185b1d43433ec1b0a0039b043ea194ef6bc0a06/model.yaml'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/models/model.py", line 221, in from_fs
with item_fs.open(MODEL_YAML_FILENAME, "r") as model_yaml:
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/wrapfs.py", line 472, in open
open_file = _fs.open(
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 643, in open
with convert_os_errors("open", path):
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/error_tools.py", line 89, in __exit__
reraise(fserror, fserror(self._path, exc=exc_value), traceback)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/six.py", line 718, in reraise
raise value.with_traceback(tb)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
return io.open(
fs.errors.ResourceNotFound: resource 'model.yaml' not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/python/yasser/n_venv/bin/openllm", line 8, in<module>sys.exit(cli())
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm_cli/entrypoint.py", line 160, in wrapper
return_value = func(*args, **attrs)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
returnf(get_current_context(), *args, **kwargs)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm_cli/entrypoint.py", line 141, in wrapper
return f(*args, **attrs)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm_cli/entrypoint.py", line 366, in start_command
llm = openllm.LLM(
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm/_llm.py", line 223, in __init__
model = bentoml.models.get(self.tag)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/models.py", line 45, in get
return _model_store.get(tag)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 149, in get
self._recreate_latest(_tag)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 118, in _recreate_latest
items = self.list(tag.name)
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 106, in list
return [self._get_item(t) fortin tags]
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 106, in<listcomp>return [self._get_item(t) fortin tags]
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 114, in _get_item
returnself._item_type.from_fs(self._fs.opendir(tag.path()))
File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/models/model.py", line 224, in from_fs
raise BentoMLException(
bentoml.exceptions.BentoMLException: Failed to load bento model because it does not contain a 'model.yaml'
Environment
#### System information
`bentoml`: 1.1.11
`python`: 3.10.9
`platform`: Linux-5.15.0-1052-aws-x86_64-with-glibc2.31
`uid_gid`: 1000:1000
<details><summary><code>pip_packages</code></summary>
<br>
</details>
_______________________________________________________________________________________________________
- `transformers` version: 4.37.2
- Platform: Linux-5.15.0-1052-aws-x86_64-with-glibc2.31
- Python version: 3.10.9
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: 0.27.2
- Accelerate config: not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
### System information (Optional)
_No response_
The text was updated successfully, but these errors were encountered:
yasserkh2
changed the title
bug:
When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process.
Feb 20, 2024
aarnphm
changed the title
When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process.
feat: supporing aya models.
Jun 3, 2024
Describe the bug
When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process.
bentoml.exceptions.BentoMLException: Failed to load bento model because it does not contain a 'model.yaml'
To reproduce
1-add the model like they describe here [https://github.com/bentoml/OpenLLM/blob/main/ADDING_NEW_MODEL.md]
2-pip install openllm
3-openllm -h
4-Run the command to add the TRUST_REMOTE_CODE=True openllm start CohereForAI/aya-101 model.
Logs
Environment
accelerate==0.27.2
aiohttp==3.9.3
aioprometheus==23.12.0
aiosignal==1.3.1
annotated-types==0.6.0
anyio==4.3.0
appdirs==1.4.4
asgiref==3.7.2
asttokens==2.4.1
async-timeout==4.0.3
attrs==23.2.0
bentoml==1.1.11
bitsandbytes==0.41.3.post2
build==0.10.0
cattrs==23.1.2
certifi==2024.2.2
charset-normalizer==3.3.2
circus==0.18.0
click==8.1.7
click-option-group==0.5.6
cloudpickle==3.0.0
coloredlogs==15.0.1
comm==0.2.1
contextlib2==21.6.0
cuda-python==12.3.0
cupy-cuda12x==12.1.0
datasets==2.17.1
debugpy==1.8.1
decorator==5.1.1
deepmerge==1.1.1
Deprecated==1.2.14
dill==0.3.8
distlib==0.3.8
distro==1.9.0
einops==0.7.0
exceptiongroup==1.2.0
executing==2.0.1
fastapi==0.109.2
fastcore==1.5.29
fastrlock==0.8.2
filelock==3.13.1
filetype==1.2.0
frozenlist==1.4.1
fs==2.4.16
fsspec==2023.10.0
ghapi==1.0.4
h11==0.14.0
httpcore==1.0.3
httptools==0.6.1
httpx==0.26.0
huggingface-hub==0.20.3
humanfriendly==10.0
idna==3.6
importlib-metadata==6.11.0
inflection==0.5.1
ipykernel==6.29.2
ipython==8.21.0
jedi==0.19.1
Jinja2==3.1.3
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter_client==8.6.0
jupyter_core==5.7.1
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib-inline==0.1.6
mdurl==0.1.2
mpmath==1.3.0
msgpack==1.0.7
multidict==6.0.5
multiprocess==0.70.16
mypy-extensions==1.0.0
nest-asyncio==1.6.0
networkx==3.2.1
ninja==1.11.1.1
numpy==1.26.4
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-ml-py==11.525.150
nvidia-nccl-cu12==2.18.1
nvidia-nvjitlink-cu12==12.3.101
nvidia-nvtx-cu12==12.1.105
openllm==0.4.44
openllm-client==0.4.44
openllm-core==0.4.44
opentelemetry-api==1.20.0
opentelemetry-instrumentation==0.41b0
opentelemetry-instrumentation-aiohttp-client==0.41b0
opentelemetry-instrumentation-asgi==0.41b0
opentelemetry-sdk==1.20.0
opentelemetry-semantic-conventions==0.41b0
opentelemetry-util-http==0.41b0
optimum==1.17.1
orjson==3.9.14
packaging==23.2
pandas==2.2.0
parso==0.8.3
pathspec==0.12.1
pexpect==4.9.0
pillow==10.2.0
pip-requirements-parser==32.0.1
pip-tools==7.3.0
platformdirs==4.2.0
prometheus_client==0.20.0
prompt-toolkit==3.0.43
protobuf==4.25.3
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==15.0.0
pyarrow-hotfix==0.6
pydantic==2.6.1
pydantic_core==2.16.2
Pygments==2.17.2
pynvml==11.5.0
pyparsing==3.1.1
pyproject_hooks==1.0.0
python-dateutil==2.8.2
python-dotenv==1.0.1
python-json-logger==2.0.7
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
pyzmq==25.1.2
quantile-python==1.1
ray==2.9.2
referencing==0.33.0
regex==2023.12.25
requests==2.31.0
rich==13.7.0
rpds-py==0.18.0
safetensors==0.4.2
schema==0.7.5
scipy==1.12.0
sentencepiece==0.2.0
simple-di==0.1.5
six==1.16.0
sniffio==1.3.0
stack-data==0.6.3
starlette==0.36.3
sympy==1.12
tokenizers==0.15.2
tomli==2.0.1
torch==2.1.2
tornado==6.4
tqdm==4.66.2
traitlets==5.14.1
transformers==4.37.2
triton==2.1.0
typing_extensions==4.9.0
tzdata==2024.1
urllib3==2.2.1
uvicorn==0.27.1
uvloop==0.19.0
virtualenv==20.25.0
vllm==0.3.1
watchfiles==0.21.0
wcwidth==0.2.13
websockets==12.0
wrapt==1.16.0
xformers==0.0.23.post1
xxhash==3.4.1
yarl==1.9.4
zipp==3.17.0
The text was updated successfully, but these errors were encountered: