You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi! Thanks for the excellent work.
While I train my own custom data using maptrv2(Lidar+Camera) model, I met the error in voxelize.py: Traceback (most recent call last): File "./tools/train.py", line 262, in <module> main() File "./tools/train.py", line 251, in main custom_train_model( File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/bevformer/apis/train.py", line 27, in custom_train_model custom_train_detector( File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/bevformer/apis/mmdet_train.py", line 200, in custom_train_detector runner.run(data_loaders, cfg.workflow) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run epoch_runner(data_loaders[i], **kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train self.run_iter(data_batch, train_mode=True, **kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter outputs = self.model.train_step(data_batch, self.optimizer, File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 52, in train_step output = self.module.train_step(*inputs[0], **kwargs[0]) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 237, in train_step losses = self(**data) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 198, in forward return self.forward_train(**kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 214, in new_func output = old_func(*new_args, **new_kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 304, in forward_train lidar_feat = self.extract_lidar_feat(points) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 128, in new_func output = old_func(*new_args, **new_kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 254, in extract_lidar_feat feats, coords, sizes = self.voxelize(points) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 214, in new_func output = old_func(*new_args, **new_kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 227, in voxelize ret = self.lidar_modal_extractor["voxelize"](res) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/Downloads/MapTR/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 144, in forward return voxelization( File "/home/ubuntu/Downloads/MapTR/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 68, in forward voxel_num = hard_voxelize( RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
And i can't find the definition of hard_voxelize that imported as below in voxelize.py: from .voxel_layer import dynamic_voxelize, hard_voxelize
Can anyone help?
The text was updated successfully, but these errors were encountered:
Hi! Thanks for the excellent work.
While I train my own custom data using maptrv2(Lidar+Camera) model, I met the error in voxelize.py:
Traceback (most recent call last): File "./tools/train.py", line 262, in <module> main() File "./tools/train.py", line 251, in main custom_train_model( File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/bevformer/apis/train.py", line 27, in custom_train_model custom_train_detector( File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/bevformer/apis/mmdet_train.py", line 200, in custom_train_detector runner.run(data_loaders, cfg.workflow) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run epoch_runner(data_loaders[i], **kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train self.run_iter(data_batch, train_mode=True, **kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter outputs = self.model.train_step(data_batch, self.optimizer, File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 52, in train_step output = self.module.train_step(*inputs[0], **kwargs[0]) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 237, in train_step losses = self(**data) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 198, in forward return self.forward_train(**kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 214, in new_func output = old_func(*new_args, **new_kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 304, in forward_train lidar_feat = self.extract_lidar_feat(points) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 128, in new_func output = old_func(*new_args, **new_kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 254, in extract_lidar_feat feats, coords, sizes = self.voxelize(points) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 214, in new_func output = old_func(*new_args, **new_kwargs) File "/home/ubuntu/Downloads/Maptr_Fish-main/maptr/projects/mmdet3d_plugin/maptr/detectors/maptrv2.py", line 227, in voxelize ret = self.lidar_modal_extractor["voxelize"](res) File "/home/ubuntu/anaconda3/envs/maptr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/ubuntu/Downloads/MapTR/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 144, in forward return voxelization( File "/home/ubuntu/Downloads/MapTR/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 68, in forward voxel_num = hard_voxelize( RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
And i can't find the definition of hard_voxelize that imported as below in voxelize.py:
from .voxel_layer import dynamic_voxelize, hard_voxelize
Can anyone help?
The text was updated successfully, but these errors were encountered: