Very laggy when reading data from device

[c0debreaker]

I converted a NodeJS code. The NodeJS program is very fluid and I don't see any lagging when I am reading from the device. Here's the NodeJS code https://github.com/fpv-wtf/voc-poc/blob/master/index.js

And here is my code. When I am using this reading from the same device, it is very laggy. What am I doing wrong?

#!/usr/bin/env python

import usb.core
import usb.util
import sys
import signal

# find our device
usb_device = usb.core.find(idVendor=0x2ca3, idProduct=0x1f)

if usb_device is None:
    raise ValueError('Device not found!')

cfg = usb_device.get_active_configuration()
intf = cfg[(3,0)]

out_endpoint = usb_device[0][(3,0)][0]
in_endpoint = usb_device[0][(3,0)][1]

# Code copied from https://stackoverflow.com/questions/67453535/i-keep-getting-usb-core-usberror-errno-16-resource-busy-when-trying-to-read
if usb_device.is_kernel_driver_active(0):
    try:
        usb_device.detach_kernel_driver(0)
    except usb.core.USBError as e:
        sys.exit("Could not detatch kernel driver from interface({0}): {1}".format(intf, str(e)))

magic = bytes.fromhex("524d5654")
out_endpoint.write(magic)

def signal_handler(sig, frame):
    # print('You pressed Ctrl+C!')
    usb.util.release_interface(usb_device, intf)
    sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)

while True:
    try:
        data = usb_device.read(in_endpoint.bEndpointAddress, in_endpoint.wMaxPacketSize)
        sys.stdout.buffer.write(data)

    except usb.core.USBError as e:
        data = None
        if e.args == ('Operation timed out',):
            continue

[Mickey70]

Have you had any success solving your problem? I'm struggling with a stubborn time-out problem when I use the script you have provided above. Maybe unrelated, but I do not know.

[c0debreaker]

No, I have not found any solutions.

[c0debreaker]

@Mickey70 I read your question again. Maybe your Auto Temp Control is still set to ON. It should be set to OFF.

[mcuee]

data = usb_device.read(in_endpoint.bEndpointAddress, in_endpoint.wMaxPacketSize)

You should use large amount of data and not one packet by packet.

[mcuee]

As pyusb does not yet support async transfer yet (#303), the next best thing is to use a larger transfer size.

Reference:
https://marc.info/?l=libusb-devel&m=131647595921270&w=4

From Tim Roberts, an USB expert and Windows driver expert:
USB is a scheduled bus. The host controller driver lays out the entire
upcoming microframe in advance, based on the requests it has in hand.
It then sends that set of requests en masse to the host controller
hardware, which performs the transfers on its own, and fires an
interrupt when the frame (or microframe) is complete. At that time, the
host controller notifies the owners of any completed requests, and sends
off the schedule for the next microframe.

So, if you send down one request and wait, the HCD will schedule your
one request, and leave the rest of the frame empty When the microframe
completes, it fires off a notification, but you will already have missed
the next microframe. Depending on the turnaround time, you might miss
several more.

It is possible to fit 5k or 6k worth of bulk transfers into a
microframe, but only if you have all of that waiting when the microframe
is scheduled. You can do that by submitting larger requests
synchronously (although that still doesn't maximize throughput), but to
get continuous streaming you must use asynchronous requests, so that
there is ALWAYS a request waiting when the HCD does its scheduling.

--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[c0debreaker]

Awesome! I'll try it once I am back in Chicago which is next week. Thank you so much for sharing all the information!!!

[c0debreaker]

I forgot to ask. Does it mean that I will have to also use the latest libusb? I forgot which version I'm using. I'm on Debian 11.

[mcuee]

libusb latest release libusb-1.0.27 is recommended but probably not necessary. Debian 11 "bullseye" comes with libusb-1.0.24 and it should be okay under Linux.
https://packages.debian.org/search?keywords=libusb-1.0-0-dev

[c0debreaker]

Hi @mcuee I just got back. I can now work on my code today. You wrote You can see that I am using 64KB transfer size.. Where in your code you wrote 64Kb?

[Mickey70]

I was going back and forth between using pyusb and pyserial because I could not get data transfers to work with either. Then, I stumbled on to a YouTube video https://www.youtube.com/watch?v=xc9rUI0F6Iw in which the creator was having the same problem as me... configuring the port went fine, but no data transfer. At specifically about 2 minutes 20 seconds into the video, the author explains what he found that made things work for him (and for me), specifically (i) force an Arduino reset by setting the DTR line low, (ii) waiting one second, (iii) flushing the Arduino input using the flushInput() command, and then finally (iv) setting the DTR line True. I found that the only serial port that would completely work on the Arduino is the ttyACM0 port; I still need to understand this better. I do not know (yet) if the same kind of thing was causing my pyusb efforts to fail or not.

[c0debreaker]

Mine wasn't failing. It was just slow. I would like to try the suggestion that mcuee told me.

[mcuee]

@Mickey70
Your device is a USB to Serial device so it is better not to use pyusb but rather things like pyserial. The high level USB to serial driver will handle the things for you and you do not need to use lower level stuff like USB Bulk Transfer using pyusb/libusb.

[c0debreaker]

data = usb_device.read(in_endpoint.bEndpointAddress, in_endpoint.wMaxPacketSize)

You should use large amount of data and not one packet by packet.

Like this code? I changed the last parameter to 64000.

data = usb_device.read(in_endpoint.bEndpointAddress, 64000)

[c0debreaker]

Oh, I see what you're saying chunkr = 2**16. This is equal to 65536 and .read method can accept 3 parameters. I'll do that :)

[c0debreaker]

YAY! It's fixed!!! Thank you so much!