LunarLander (PPO)
- Approach: Proximal Policy Optimization (PPO)
- Learning: Hyperparameter sensitivity, training stability, iteration workflow
- Tracking: TensorBoard for reward curves and training diagnostics
- Page: details
- Repo: Coming soon
Hello, I'm Carlos!
CS @ UTRGV · Graduating Fall 2026 · Currently: RL (PPO)
I am a Computer Science Student at the University of Texas of the Rio Grande Valley (UTRGV).
I enjoy running, watching sci-fi, fantasy movies, playing videogames, and spending time with my family, friends and my dog Maya.
On this website you will be able to learn about me, read about recent developments of my life
and see my most recent project and its progress.
Interests & Hobbies
- Running
- Sci-Fi & Fantasy
- Videogames
- Time with Family & Friends
Current goals: keep learning Reinforcement Learning, start an RL research project, brainstorm Senior Project research questions, and finish reading The Count of Monte Cristo.
Carlos' Coding Journey
I started learning programming in my senior year of high school, and I was eager to keep going, so I applied to my local university as a Computer Science major.
So far, this journey has come with a lot of challenges, but I have no regrets. I'm going to do my best to have fun and keep learning.
Now
- Learning: Reinforcement Learning and training agents (PPO)
- Research: Attending UTRGV Machine/AI Lab meetings and learning how to read papers
- Building: Setting up a consistent experimentation workflow with TensorBoard
Projects
CartPole (Gymnasium)
- Status: Starting next
- Goal: Build intuition for policy learning and evaluation
- Focus: Learning curves, variance across seeds, stability
- Page: details
- Repo: Coming soon
Experiment Log
| Environment | Algorithm | Timesteps | n_steps | Batch | Result |
|---|---|---|---|---|---|
| LunarLander | PPO | 3M | 2048 | 128 | Mean reward 297.73 ± 15.27 (8 epochs) |
AI/ML Journey
- Sep 2025 - Started Online Course CS 188 Berkeley - Introduction to Artificial Intelligence.
- Nov 2025 - Finished MIT algorithms book and took a special interest in Deep Learning section of the book.
- Jan 2026 - Now - Following the Hugging Face Reinforcement Learning Course, attending RL research meetings, and getting hands-on practice training PPO agents.
- Next - Start the CartPole environment (Gymnasium) and keep building a consistent experimentation workflow with TensorBoard.
Updates
Update: February 26, 2026
RL research · UTRGV lab meetings · Senior project planning
Lately I've been getting more interested in the research side of Machine Learning, especially Reinforcement Learning and what it takes to train agents through RL.
I've also started taking initiative and attending the Machine Learning meetings my professor holds for the Machine/AI Lab at UTRGV. I've been taking advice from my professors as well as PhD and Masters students on how to get started, what to focus on, and how to build a stronger foundation for understanding ML and research as a whole.
I also finally got a team for my Senior Project, which I'll be doing in my last semester of University (next semester). For our Senior Project we want to take a research approach and write and publish a paper on Reinforcement Learning and Federated Learning.
Update: November 04, 2025
Last year of school · AI courses · SWE project
WOW Almost a whole year lol (great going me from the past). I just wanted to update you all with my life update.
Everything is going great, I am in my last semesters of University, I have an expected graduation date for Fall 2026, so in 1 year I will be graduating. This is both exciting and scary to think about.
I have sign up to my next set of courses, now I will be diving head-first (lol) into AI related Courses. I have registered for Automata, AI Application Agents, and Digital Visions. I honestly can't wait to get started on them.
Right now I am in a SWE Class where we were tasked to build our own web-application, and I am one of the lead Developers (WOOP WOOP). So yeah, expect more from me soon in the next update.
Update: December 16, 2024
Finals done · Planning next semester · Website updates
Finals week was last week, now I am officially out of school, I ended up passing all of my classes.
Right now my main focus is to prepare for my next set of classes and what type of topics the class will cover.
I will continue to also focus on working on this website and personal projects as I have already started a CSS course, and also a JavaScript course.
Update: December 05, 2024
Semester wrap-up · Website work · Icons
My semester just finished yesterday (Dec 04) and the last thing i have to do is just a security project.
After that I will be working now more on this Personal Website and Personal Projects.
I also added Icons to the website, so the LIFE UPDATE section has icons, and also the title page.
Update: May 20, 2024
Back to coding · Website improvements · Personal projects
I got out of school for at least 2 weeks now, and had been on a break from coding. I decided to start working again on the website.
I have realized that it is time for me to start getting serious and lock in (lol) but anyways I will add a JavaScript to the folder so that I can decorate this website to my liking.
Update on application: I am still working on it, just came to a mental block on what steps to take next, but I'll be fine, I just have to devote my attention to it. Will update when something interesting happens.
Update: April 14, 2024
Python · Desktop app · Password manager
I started developing a desktop application using PyQt6, basically desktops development but with Python
The application is a password manager, something basic so I can get started on what I want to do