Issue when running evaluation shell script

[milan5000]

I have followed the instructions on the README file, trying both the manual installation (with recommended specs) and the Docker container. I configured the project with my OpenAI API key following the instructions. Then I launched Carla and ran the shell script provided. I can verify that everything up to this point worked correctly and I have checked that I followed all preceding instructions properly.

Initially, I received the following error:

simulation/leaderboard/leaderboard/leaderboard_evaluator_parameter.py:24: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
 import pkg_resources
Traceback (most recent call last):
 File "simulation/leaderboard/leaderboard/leaderboard_evaluator_parameter.py", line 43, in <module>
   from leaderboard.scenarios.route_scenario import RouteScenario
 File "/root/langcoop/simulation/leaderboard/leaderboard/scenarios/route_scenario.py", line 52, in <module>
   from leaderboard.sensors.fixed_sensors import TrafficLightSensor
 File "/root/langcoop/simulation/leaderboard/leaderboard/sensors/fixed_sensors.py", line 16, in <module>
   from team_code.utils.map_drawing import \
 File "/root/langcoop/simulation/leaderboard/team_code/utils/__init__.py", line 1, in <module>
   import carla_birdeye_view
 File "/root/miniconda3/envs/LangCoop/lib/python3.8/site-packages/carla_birdeye_view/__init__.py", line 13, in <module>
   from carla_birdeye_view.mask import (
 File "/root/miniconda3/envs/LangCoop/lib/python3.8/site-packages/carla_birdeye_view/mask.py", line 5, in <module>
   from carla_birdeye_view import lanes
 File "/root/miniconda3/envs/LangCoop/lib/python3.8/site-packages/carla_birdeye_view/lanes.py", line 4, in <module>
   from cv2 import cv2 as cv
ImportError: Bindings generation error. Submodule name should always start with a parent module name. Parent name: cv2.cv2. Submodule name: cv2

I accessed the relevant Python files and changed the formatting for the cv2 import statements to work correctly. Afterwards, I get the following output instead:

simulation/leaderboard/leaderboard/leaderboard_evaluator_parameter.py:24: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
 import pkg_resources
results/results_driving_speed_curvature_CoT_concise_image_intent_2agent/v2x_final/town05_short_collab/r_partial_repeat0/ego_vehicle_0/results.json do not exists, continue!

At which point the program freezes and does not respond, even to manually stopping the process with ctrl+C. From my own checking, it seems like the results.json file, as well as other data in the results/ directory, is not generated correctly. I do not see information anywhere in the documentation about how to resolve this issue. Any help would be greatly appreciated, thank you for your time.

[XiangboGaoBarry]

Thank you for your interest in our work!
This issue may be caused by using an unstable OpenRouter API. Some APIs are known to hang indefinitely. We plan to improve the robustness of the code in future updates. At this time, please try using a more stable API such as gpt-4o-mini or claude-sonnet-4.
Let me know if this resolves the issue. If not, I can try to reproduce and debug it later.

[milan5000]

Thank you for your reply, I've tried to use gpt-4o-mini and it has produced the same result. I believe I have kept the default configuration for gpt-4o-mini, and I've confirmed my API key works. I've currently put a pause on replicating the project for now, so there's no need to continue troubleshooting; thank you again for your help.

[Kroery]

I encountered the same issue when running experiments with the Qwen/Qwen2.5-VL-3B-Instruct-AWQ model. After modifying the cv2 import statements to fix the import error, I got the following output:

results/results_driving_speed_curvature_CoT_concise_image_intent_2agent/v2x_final/town05_short_collab/r_partial_repeat0/ego_vehicle_0/results.json do not exists, continue!

After this, the program freezes and becomes unresponsive. Could you please advise if there is a solution for this?

[XiangboGaoBarry]

@Kroery @milan5000 Thanks for reaching out! and sorry for the confusion.

We have investigated the issue, and the freezing occurs because the Carla_port=xxxx specified in the bash file does not match the --world-port=xxxx used when starting the Carla Simulator.

I’ve updated the default --world-port to 2002. If you follow the new command, everything should work as expected.