open source contributions by me →

Hugging Face TRL
Issue was around DPOTrainer does not work for multimodal Gemma 4.
Hugging Face Gym hil
Issue was why is the obs of gym_hil the same each time it is reset?