After searching why Steam never closed in the tray icon area like it does on Windows, I found an article talking about the issue, issue which has been reported in 2013.
There is a fix, and it is really simple. You just need to set the STEAM_FRAME_FORCE_CLOSE=1 environment variable before starting Steam
First step, I opened the ~/.local/share/applications/ folder and got rid of the steam.desktop file (mine was a symlink to /usr/lib/steam/steam.desktop, so instead of modifying the original file in /usr/lib/steam/, I deleted the symlink in ~/.local/share/applications/).
Then I created a new steam.desktop file in ~/.local/share/applications/ and opened it in Kate.
I pasted the content of the original steam.desktop file and edited it to change the startup command line and to remove the unnecessary application options (Big Picture, Store, and so on..).
[Desktop Entry]
Name=Steam
Comment=Application for managing and playing games on Steam
Comment[fr]=Application de gestion et d'utilisation des jeux sur Steam
Exec=STEAM_FRAME_FORCE_CLOSE=1 /usr/bin/steam %U
Icon=steam
Terminal=false
Type=Application
Categories=Network;FileTransfer;Game;
MimeType=x-scheme-handler/steam;
Actions=KF2;DRG;_SEPARATOR_;Library;Friends;Servers;Settings;_SEPARATOR_;EXIT;_SEPARATOR_;
[Desktop Action KF2]
Name=Killing Floor 2
Exec=steam steam://run/232090
[Desktop Action DRG]
Name=Deep Rock Galactic
Exec=steam steam://run/548430
[Desktop Action Library]
Name=Library
Name[fr]=Bibliothèque
Exec=steam steam://open/games
[Desktop Action Friends]
Name=Friends
Name[fr]=Amis
Exec=steam steam://open/friends
[Desktop Action Servers]
Name=Servers
Name[fr]=Serveurs
Exec=steam steam://open/servers
[Desktop Action Settings]
Name=Settings
Name[fr]=Paramètres
Exec=steam steam://open/settings
[Desktop Action EXIT]
Name=Exit
Name[fr]=Quitter
Exec=steam -exitsteam
After that Steam launches with command STEAM_FRAME_FORCE_CLOSE=1 /usr/bin/steam %U, it does not have all the bloat options anymore when you right click the application window icon in the taskbar, and it properly closes to tray as expected.