Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Carla FollowLane Q-Learn with red segmentation image #172

Open
wants to merge 67 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
d673526
fixed windows size, print messsages
pjfernandecabo Mar 6, 2023
b0a504e
first training steps: fixing distance to center and clipping Throttle
pjfernandecabo Mar 7, 2023
aa3b478
first correct training
pjfernandecabo Mar 9, 2023
9c6ff82
seizing PR: adding image png for README
pjfernandecabo Mar 9, 2023
744e391
eliminating displaymanager
pjfernandecabo Mar 13, 2023
3416e82
fixed carla server disconnection, changing displaymanager for cv2
pjfernandecabo Mar 14, 2023
477ae79
adding new sensors lane changing and obstacles, and changing to town 07
pjfernandecabo Mar 15, 2023
26a2eb3
Updated color mask for Town07
sergiopaniego Mar 16, 2023
034694f
BEV central line mask for simplified perception
sergiopaniego Mar 16, 2023
a989808
Updated BEV mask with distance to center
sergiopaniego Mar 16, 2023
ec42329
Merge pull request #180 from JdeRobot/issue-179
pjfernandecabo Mar 16, 2023
575b446
town04 fixing rgb mask
pjfernandecabo Mar 17, 2023
1fb9429
adding points and info in mask image, and modifying mask threshold to…
pjfernandecabo Mar 18, 2023
fba0077
training many epochs in town07, changing qlearn algorithm, distance t…
pjfernandecabo Mar 22, 2023
975cce0
fixing state_0, throttle and steering
pjfernandecabo Mar 23, 2023
29f18fc
Mask updated for Town04
sergiopaniego Mar 24, 2023
ae2ceb2
Merge pull request #182 from JdeRobot/issue-181
sergiopaniego Mar 24, 2023
b5a6896
adding alternate pose in mountain circuit Town07 with 12 poses in lan…
pjfernandecabo Apr 3, 2023
052221a
Merge branch 'carla_followlane_straight_line' of https://github.com/J…
pjfernandecabo Apr 3, 2023
d4e52f8
Merge branch 'main' into carla_followlane_straight_line
pjfernandecabo Apr 3, 2023
20b0f9f
adding town04 config
pjfernandecabo Apr 5, 2023
f7ec80a
changing weakref in callbacks
pjfernandecabo Apr 17, 2023
f93422c
adding Carla server listener and town07_opt
pjfernandecabo Apr 25, 2023
10a4a4e
follow right lane only, changing rewards, fix dist variable
pjfernandecabo Apr 25, 2023
9b84c23
changing qlearn algorithm, adding sensor collision to replicate sgeme…
pjfernandecabo May 3, 2023
824c8d2
Updated Segmentation camera FOV to be the same as the frontal camera
sergiopaniego May 4, 2023
f8eb9b5
changing to lane center with 2 lines, right and left. Adding circuit …
pjfernandecabo May 5, 2023
17a99ca
adding a controller in throttle. Problema followlane solved!
pjfernandecabo May 8, 2023
0718cf7
remove spectator, other minor changes
pjfernandecabo May 10, 2023
bded4bb
adding inference files
pjfernandecabo May 10, 2023
ce1b722
adding stats intrinsic and extrinsic metrics
pjfernandecabo May 12, 2023
cf66b74
ending stats for QLearning
pjfernandecabo May 15, 2023
da4743e
adding plot stats files, and automatic_control autopilot
pjfernandecabo May 19, 2023
ea18cb1
town04 only right line perception
pjfernandecabo Jun 21, 2023
c04b733
adding LaneDetector
pjfernandecabo Jun 30, 2023
6f43359
ddpg carla followlane
pjfernandecabo Sep 26, 2023
5c0e0d4
adding DQN Follow Lane in Carla
pjfernandecabo Sep 27, 2023
4f06f3b
adding stable-baselines3: multiple algorithms and multiprocessing. Fi…
pjfernandecabo Sep 28, 2023
14857ac
fixed CarlaEnv in Gym
pjfernandecabo Sep 29, 2023
d7cdd0d
fixed sync mode YEAHH
pjfernandecabo Oct 10, 2023
5bb7af4
sync nd reset working
pjfernandecabo Oct 11, 2023
41b1a39
ok sync ready up to STEP()
pjfernandecabo Oct 11, 2023
6cdb09a
solved sync with step
pjfernandecabo Nov 3, 2023
72ce2fd
solving DQN with perception
pjfernandecabo Nov 6, 2023
df64202
solved central vs lateral states and drawing them
pjfernandecabo Nov 8, 2023
1611bb6
DQN sync Lane
pjfernandecabo Jan 18, 2024
53c41a9
refactoring DQN SYNC + laneDetector with regression in step() method
pjfernandecabo Jan 20, 2024
07eb0fc
finish LaneDetector in step, reward and drawing
pjfernandecabo Jan 23, 2024
dba7b58
adding step_only_right_line() method
pjfernandecabo Jan 24, 2024
96795b3
changing LaneDetector, reset() to include new states, step() centers …
pjfernandecabo Feb 9, 2024
46464bd
adding heading, function reward (center, vel, heading) and states nor…
pjfernandecabo Feb 10, 2024
9525bf7
new recurrent DQN net with variable inputs and masking
pjfernandecabo Feb 12, 2024
0bccee4
fully integrated DQN in RL-S
pjfernandecabo Feb 14, 2024
fc53078
fixed DQN NN with 15 states, LAneDetector perception, rewards adjusted
pjfernandecabo Feb 17, 2024
d1deaa9
fixed loggers. DQN First training sessions
pjfernandecabo Feb 19, 2024
127798c
refactor agents/auto_carla...faster
pjfernandecabo Feb 25, 2024
6375956
refactor CarlaEnv...If garbage collector does not work...python stincks
pjfernandecabo Feb 26, 2024
f1e61de
adding concatenate neural net imput
pjfernandecabo Feb 27, 2024
02122e0
killing Carla server at last
pjfernandecabo Feb 27, 2024
ebfa5ab
adding all to run in Landau and mypc in parallel
pjfernandecabo Feb 28, 2024
e396d0d
adding Pythorch LaneDetector model
pjfernandecabo Feb 28, 2024
9dc9a59
small changes in Landau for multi-client/multi-server
pjfernandecabo Mar 1, 2024
ef1a4da
adding collision, lane changind, obstacle sensors
pjfernandecabo Mar 4, 2024
e5f6220
adding killing python script, 2 positions town04
pjfernandecabo Mar 5, 2024
08d3cdd
dqn in both ways
pjfernandecabo Mar 5, 2024
4e6220b
changing finish lines town04 lateral
pjfernandecabo Mar 5, 2024
3ee0d5d
remove an input in step
pjfernandecabo Mar 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions rl_studio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ There are several config files to take as example. If you need more information

The following graph shows a conceptual diagram of the operation of RL-Studio in training mode. In the case of making inference or retraining, the process is similar

![](../rl_studio/docs/rls-diagram.svg)
![](../rl_studio/docs/rls-diagram.png)



# Run RL Studio
Expand Down Expand Up @@ -92,6 +93,7 @@ Remaining params should be adjusted too. There are many working yaml files in co
> need extend the rl-studio.py to listen for inputs in a port and execute the loaded brain/algorithm to provide
> outputs in the desired way.


More info about how to config and launch any task, please go to [agents](agents/README.md) section.


Expand All @@ -109,8 +111,3 @@ Launch it with `python plot_waypoints.py -t <town>` where `<town>` could be Town
![](../rl_studio/docs/Town01_topology.png)







86 changes: 82 additions & 4 deletions rl_studio/agents/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@

class TrainerFactory:
def __new__(cls, config):

"""
There are many options:

Expand Down Expand Up @@ -53,10 +52,59 @@ def __new__(cls, config):
agent=agent,
framework=framework,
)

# =============================
# Follow Lane - AutoCarla - Carla - Stable-baselines3
# =============================
if (
task == TasksType.FOLLOWLANECARLA.value
and agent == AgentsType.AUTOCARLA.value
# and algorithm == AlgorithmsType.DQN.value
and simulator == EnvsType.CARLA.value
and framework == FrameworksType.STABLE_BASELINES3.value
):
from rl_studio.agents.auto_carla.train_followlane_carla_sb3 import (
TrainerFollowLaneAutoCarlaSB3,
)

return TrainerFollowLaneAutoCarlaSB3(config)

# =============================
# FollowLane - F1 - qlearn - Carla
# Follow Lane - AutoCarla - DQN - Carla - TF
# =============================
if (
task == TasksType.FOLLOWLANECARLA.value
and agent == AgentsType.AUTOCARLA.value
and algorithm == AlgorithmsType.DQN.value
and simulator == EnvsType.CARLA.value
and framework == FrameworksType.TF.value
):
from rl_studio.agents.auto_carla.train_followlane_dqn_carla_tf import (
TrainerFollowLaneDQNAutoCarlaTF,
)

return TrainerFollowLaneDQNAutoCarlaTF(config)

# =============================
# Follow Lane - AutoCarla - DDPG - Carla - TF
# =============================
elif (
task == TasksType.FOLLOWLANECARLA.value
and agent == AgentsType.AUTOCARLA.value
and algorithm == AlgorithmsType.DDPG.value
and simulator == EnvsType.CARLA.value
and framework == FrameworksType.TF.value
):
from rl_studio.agents.auto_carla.train_followlane_ddpg_carla_tf import (
TrainerFollowLaneDDPGAutoCarlaTF,
)

return TrainerFollowLaneDDPGAutoCarlaTF(config)

# =============================
# FollowLane - AutoCarla - qlearn - Carla
# =============================
elif (
task == TasksType.FOLLOWLANECARLA.value
and agent == AgentsType.AUTOCARLA.value
and algorithm == AlgorithmsType.QLEARN.value
Expand All @@ -71,7 +119,7 @@ def __new__(cls, config):
# =============================
# FollowLine - F1 - qlearn - Gazebo
# =============================
if (
elif (
task == TasksType.FOLLOWLINEGAZEBO.value
and agent == AgentsType.F1GAZEBO.value
and algorithm == AlgorithmsType.QLEARN.value
Expand Down Expand Up @@ -340,7 +388,6 @@ def __new__(cls, config):

class InferencerFactory:
def __new__(cls, config):

agent = config["settings"]["agent"]
algorithm = config["settings"]["algorithm"]
task = config["settings"].get("task")
Expand All @@ -355,6 +402,37 @@ def __new__(cls, config):
framework=framework,
)

# =============================
# Follow Lane - AutoCarla - DDPG - Carla - TF
# =============================
if (
task == TasksType.FOLLOWLANECARLA.value
and agent == AgentsType.AUTOCARLA.value
and algorithm == AlgorithmsType.DDPG.value
and simulator == EnvsType.CARLA.value
and framework == FrameworksType.TF.value
):
from rl_studio.agents.auto_carla.inference_followlane_ddpg_carla_tf import (
InferencerFollowLaneDDPGAutoCarlaTF,
)

return InferencerFollowLaneDDPGAutoCarlaTF(config)

# =============================
# FollowLane - AutoCarla - qlearn - Carla
# =============================
if (
task == TasksType.FOLLOWLANECARLA.value
and agent == AgentsType.AUTOCARLA.value
and algorithm == AlgorithmsType.QLEARN.value
and simulator == EnvsType.CARLA.value
):
from rl_studio.agents.auto_carla.inference_followlane_qlearn_carla import (
InferencerFollowLaneQlearnAutoCarla,
)

return InferencerFollowLaneQlearnAutoCarla(config)

# =============================
# FollowLine - F1 - qlearn - Gazebo
# =============================
Expand Down
Loading