Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deque with JSONDisk throws TypeError: a bytes-like object is required, not 'int' #291

Open
BubuOT opened this issue Sep 19, 2023 · 1 comment

Comments

@BubuOT
Copy link

BubuOT commented Sep 19, 2023

I tried using a Deque like this:

Deque.fromcache(Cache(eviction_policy="none", disk=JSONDisk)) and while I'm not yet entirely sure if that's a useful thing (I guess some of the serialization caveats also apply to Deque's even if there's no key to serialize?) it probably shouldn't crash like this:

>>> deque = Deque.fromcache(Cache(eviction_policy="none", disk=JSONDisk))
>>> deque += range(200)
>>> deque[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/persistent.py", line 204, in __getitem__
    return self._index(index, self._cache.__getitem__)
  File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/persistent.py", line 161, in _index
    for key in self._cache.iterkeys():
  File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/core.py", line 2252, in iterkeys
    yield _disk_get(key, raw)
  File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/core.py", line 360, in get
    return json.loads(zlib.decompress(data).decode('utf-8'))
TypeError: a bytes-like object is required, not 'int'
@grantjenks
Copy link
Owner

That is a bug. Unfortunately, it doesn't have a great fix.

Deque uses the Cache's push() and pull() API which uses special cache keys that don't play nice with the key-serialization done in JSONDisk.

Here's the workaround:

In [6]: class JSONDiskValue(dc.JSONDisk):
   ...:     put = dc.Disk.put
   ...:     get = dc.Disk.get

In [7]: deque = dc.Deque.fromcache(dc.Cache(eviction_policy="none", disk=JSONDiskValue))
In [11]: deque += range(200)
In [12]: deque[0]
Out[12]: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants