A reinforcement-learning agent trains on my PC to drive a GT3 car around Monza in Assetto Corsa. Every lap it completes is a telemetry file, and I wanted anyone, technical or not, to be able to follow the learning from anywhere. This project is the cloud platform around that agent. Lap data streams to Amazon S3, and a Streamlit dashboard on a small EC2 instance visualises it live. A Llama 3.1 model translates each run into plain English.
Two constraints shaped the design from the start. All the heavy compute, meaning training and the language model, runs locally on my RTX 3080. The cloud side is deliberately as light as possible, running on a t4g.micro.
The training code is built on the open-source Assetto Corsa Gym benchmark (SAC / DisCor), which I credit and did not write. My work is the platform around it: the incremental uploader, the S3 data layout, the dashboard and its performance, the private LLM link, session scoping, the deploy path and the security fixes along the way.
Training the agent and running the LLM both happen on my own RTX 3080. There are no GPU instances to rent and no per-token API fees, and the telemetry and AI conversations stay on my own hardware. The cloud only stores and displays results.
The dashboard runs on a t4g.micro (2 vCPU, 1 GiB RAM, Arm-based Graviton). It never trains or runs a model. It only reads Parquet files and draws charts, which is why efficiency mattered so much. The instance is also started on demand by the launch script, so it only runs while I’m training.
The model is prompted to act as an engineer explaining the run in everyday language. Each debrief is a short, two-sentence summary that compares the run with earlier ones, without telemetry jargon. The chat answers questions in the same plain style, using a rolling memory of the last 10 runs to explain whether the driver is improving, struggling or repeatedly hitting walls.
main and restarts the app. No deploy credential has to be stored in GitHub, and a push to main is the deploy.The agent is paid for going fast, but only while it stays close to a reference racing line, and it can be charged a small penalty for jerky inputs. It is never told where to brake or turn in. It discovers that by finding out which behaviour keeps the reward stream going.
r = speed_kmh # faster is better
r *= 1.0 - abs(gap) / 12.0 # fades to zero 12 m from the racing line
r /= 300.0 # normalise
r -= action_difference_penalty * coef # optional: discourage twitchy inputsThe dashboard shows the numbers, but most people can’t read a speed trace. So next to the charts sits an AI “race engineer” that explains what the driver is doing in plain English. It has two parts, an automatic debrief for each run and a chat, and both run on Llama 3.1 on my own GPU.
Pressing Generate AI Debrief summarises the selected run. Python first gathers the facts: lap time, peak and average speed, maximum cornering G, whether the car left the track, and the Monza corner where most off-track frames happened. That corner comes from a lookup table that maps the car’s progress around the lap to named sections of the circuit. It also adds a rolling memory of the previous 10 runs, so the model can say whether things are improving.
The prompt casts the model as an R&D engineer and asks for exactly two sentences in everyday language. It forbids filler openers such as “Sure, here is a summary”, calls the agent “the driver”, requires specific figures, and limits it to the single most important corner. The result is saved on the server beside the run, so it appears instantly next time.
Visitors can ask the engineer anything about the session, such as “is it getting faster?” or “where does it keep crashing?”. Each question builds the same fact block as the debrief, adds a system prompt that asks for short, friendly, non-technical answers with exact numbers and real corner names, and sends the last 10 messages of the conversation so follow-up questions make sense.
Replies stream back token by token, so the first words appear almost immediately rather than after the whole answer is written. The model also stays loaded on the GPU for 30 minutes between questions, so follow-ups don’t pay the model-loading delay again.
The model is never asked to calculate anything. Python computes every statistic from the Parquet data, and the prompt hands those figures over as plain text. The model’s job is only to turn them into a readable explanation, and it is instructed to quote the exact figures it was given. The arithmetic stays in code, where it is reliable.
The model lives on my PC, which is sometimes off. A cached health check every 20 seconds shows Engineer online or Engineer offline under the chat, calls use a timeout long enough to cover a cold model load, and any error becomes a friendly message in the chat instead of a broken page. The chat is also drawn first and never waits on S3, so it is available the moment the page loads.
Example debrief from the live dashboard “The driver is currently learning and improving its driving skills. In the last few runs, the driver’s peak speed has increased from 6.0 m/s to 20.7 m/s, and it has completed a lap in 1 minute and 41 seconds, but it still goes off-track, particularly at Parabolica corner, and its average speed is around 9.0 m/s.”
Both the debrief and the chat receive the same block, built fresh from the telemetry each time they are used.
[ACTIVE RUN METRICS]
- Total Training Runs So Far: {len(runs)}
- Completed Lap Time: {lap_str}
- Peak Speed: {speed.max():.1f} m/s (Average Speed: {speed.mean():.1f} m/s)
- Max Cornering G-Force: {g:.2f} G
- Went Off Track: {off_track} (Primary Off-Track Corner: {corner})
[LAST 10 RUNS ROLLING MEMORY]
{history}Tokens are streamed from Ollama on my PC to the EC2 app and straight to the page, with the model kept warm between questions.
def stream_reply(messages):
for chunk in ollama_client().chat(
model=OLLAMA_MODEL, messages=messages,
stream=True, keep_alive="30m"):
yield chunk["message"]["content"]
# the chat sends the system prompt plus the last 10 messages
reply = st.write_stream(stream_reply([system_prompt] + history))
The agent uses reinforcement learning: it drives, gets a score for how well it drove, and gradually adjusts its behaviour to score higher. The algorithm is Soft Actor-Critic (SAC) with an optional DisCor correction. It comes from the open-source Assetto Corsa Gym benchmark (Remonda et al., NeurIPS 2024), whose SAC code builds on toshikwa/discor.pytorch. I didn’t write it. I ran it on my own machine and built the tooling to watch it learn, so I needed to understand it well enough to read what the dashboard was telling me.
Twenty-five times a second the agent receives telemetry, such as speed and position on the track, a short history of recent frames, and look-ahead features describing the curvature and target speed of the road ahead. It outputs three numbers between −1 and 1: steering, throttle and brake. These are relative nudges with rate limits rather than absolute positions, so the wheel can’t snap from lock to lock in one tick. An episode ends if the car leaves the track, or if it stays slower than 1 m/s for 10 seconds.
An actor network picks the controls as a probability distribution, so sampling from it makes the car try slightly different things. Two critic networks predict how much reward will follow from a situation and an action, and the lower of the two estimates is used so the agent doesn’t become over-optimistic. Slow-moving copies of the critics are nudged 0.5% toward the real ones each step, which keeps the learning targets stable. An entropy bonus, tuned automatically, rewards staying a little unpredictable: the agent explores widely early on and commits as it learns.
Learned values are built from other learned values, so errors can compound. DisCor trains extra error networks that estimate how unreliable each training target is, then down-weights the unreliable samples so the critics concentrate on data they can trust. It is switched on with --algo discor, and the default run uses plain SAC.
The first 2,000 steps are random, only to fill a replay buffer that can hold up to 8 million steps. From then on the agent learns while it drives, from random batches of 128 remembered steps. Because it reuses old experience, it learns from far less real-time simulation than methods that discard data after one use. It looks 3 steps ahead when scoring decisions, with a discount of 0.992 that reaches about five seconds into the future. Checkpoints are saved regularly, and evaluation uses the mean action with no exploration noise. Every episode is written out as a Parquet file, which is where my pipeline begins.
Assetto Corsa is built for human players. It has no official way for a program to send steering and pedal commands. The benchmark’s answer is vJoy, a virtual joystick driver for Windows. To the game, vJoy looks like a real controller. To my code, it is a set of axes that can be set from Python. Getting that bridge working on my own machine was a prerequisite for everything else on this page.
sensors_par plugin inside the game streams telemetry back to the client over TCP, and the loop repeats.A small Python app called sensors_par runs inside Assetto Corsa. It opens TCP sockets: one that streams the car’s state at 25 Hz, downsampled from the game’s 50 Hz tick, and one for simulation management, such as resetting the car and fetching track and car information. The Gym environment connects as a client, so the agent’s observations arrive over a local network connection.
Each step, the client converts the agent’s three outputs into joystick axis positions: steering from the range −1 to 1 onto 0 to 2, and each pedal onto 0 to 1. It then writes them to the vJoy device through the driver’s DLL with ctypes. Assetto Corsa is set to use the vJoy device as its controller, so as far as the game is concerned, a steering wheel and pedals are being moved.
Installing the vJoy driver, copying the plugin into the game’s apps/python folder, installing the controller preset and socket libraries, and installing the Custom Shaders Patch through Content Manager so the car can be restarted in place. Then, in the game: enable the app, select vJoy as the controller, run a Hotlap session, switch ABS, traction and stability control off with automatic gearbox on, and cap the frame rate at 50 FPS to match the 50 Hz tick.
My PC runs the game, the training process and the language model, all on the same RTX 3080. vJoy commands are therefore applied locally by the training process (vjoy_executed_by_server: False, 50 Hz sampling). The benchmark also supports running the game on one machine and the agent on another, with vJoy applied on the game machine at 100 Hz, which I didn’t need.
The agent works in the range −1 to 1. The virtual joystick expects positive axis ranges, so every command is shifted and clamped before it reaches vJoy.
self.steer = self.steer_cmd + 1 # [-1, 1] -> [0, 2], clamped
self.acc = (self.pedal_cmd + 1) / 2 # [-1, 1] -> [0, 1], clamped
self.brake = (self.brake_cmd + 1) / 2 # [-1, 1] -> [0, 1], clamped
# vJoy driver DLL, loaded with ctypes
self.dll = ctypes.CDLL("C:\\Program Files\\vJoy\\x64\\vJoyInterface.dll")
self.dll.AcquireVJD(self.reference) # claim the virtual joystickUploads each lap exactly once, ignores files that are still being written, and reports failures instead of swallowing them. Keeping S3 timestamps meaningful is what lets the dashboard tell whether a run is live.
if time.time() - st.st_mtime < SETTLE_SECONDS:
continue # still being written
sig = (st.st_size, st.st_mtime_ns)
if uploaded.get(path) == sig:
continue # unchanged since last upload
key = "outputs/" + os.path.relpath(path, LOCAL_DIR).replace(os.sep, "/")
s3.upload_file(path, BUCKET_NAME, key)
uploaded[path] = sigReads only the columns a chart needs and keys the cache on the file’s modification time. Finished laps are served from memory, and a lap still being written is refreshed.
def read_cols(path, wanted):
with get_fs().open(path, "rb") as f:
pf = pq.ParquetFile(f)
names = set(pf.schema_arrow.names)
cols = [c for c in wanted if c in names]
return pf.read(columns=cols).to_pandas()
@st.cache_data(ttl=300, show_spinner=False, max_entries=64)
def load_run(path, mtime):
# mtime is part of the cache key: a run still uploading is re-read, finished runs are not
return read_cols(path, NEEDED_COLS)Recordings from training, in the order they were captured, from the first steps to a full lap. Click a clip to pause it.
The earliest footage in the series: the first time the model is let loose on Monza behind the wheel of a GT3 car. The sim flags a track cut early on.
Before corners come the basics. The agent works on holding a steady line down a straight.
A corner the agent hasn’t worked out yet.
The first run at Turn 1. The sim flags a track cut, one of the mistakes the agent has to learn to avoid.
Another go at Turn 1 as the agent keeps iterating.
A short clip of the agent learning to keep the car on the tarmac.
The first attempt at the Ascari chicane.
A second run through Ascari.
A third run through Ascari.
The fourth attempt at the chicane.
Back to Turn 1 for another go.
A complete lap of Monza, start to finish, driven by the trained agent.
The original dashboard re-listed the bucket and re-read whole Parquet files on every refresh, and the AI chat sat behind the slow work so it never appeared. I collapsed the listing into one cached call, pruned reads to the needed columns, and moved heavy statistics into a background thread pool. I also rendered the chat first so it appears immediately. On a 1 GiB instance there was no headroom to brute-force it, so the fix was efficiency, which is why it now runs comfortably on the smallest Graviton size.
Even after the code fixes, the live site kept resetting. The cause was on the server: the deploy watchdog tracked a master branch while the repository used main, so every pull failed and the app restarted every 30 seconds. I found it by comparing process uptimes with the watchdog log, corrected the script, and confirmed the app then stayed up.
The first sync script re-uploaded every file every 10 seconds, which saturated the upload link and reset every object’s last-modified time, so the dashboard could not tell which run was newest or whether a simulation was running. Switching to change-detected, settle-aware uploads restored trustworthy timestamps and made the live indicator possible.
A private key had ended up in the repository. I rotated the EC2 key pair, replaced the server’s authorized keys, audited login history and found no unauthorised access, and added ignore rules so keys can’t be committed again. I then retired the CI step that deployed over SSH in favour of the pull-based watchdog, so no deploy key is stored in GitHub at all.
Keeping inference on my own 3080 while serving a public dashboard from AWS meant connecting the two safely. Ollama was only listening on loopback, so I rebound it and linked the machines over Tailscale instead of exposing a port to the internet. The dashboard checks the model’s status, streams replies, and carries on working if the PC is switched off.
Run the dashboard and watchdog as systemd services so they survive a reboot. Define the S3 bucket, EC2 instance and security group in Terraform. Restrict SSH in the security group to a single trusted address, or remove it in favour of SSM Session Manager.