Entering a Quant Competition with No Experience: Learning as I Went

In early October, I entered my university’s quantitative finance competition, CSI300 Challenge 2026. The task was to score all 300 constituents of the CSI 300 each day and see who could rank them most accurately. I had no experience with quant competitions, so I relied on AI assistants: I set requirements, made decisions, and asked questions; AI handled interpreting the rules, training, packaging, and submission. In other words, vibe coding—with me learning along the way. I started with Codex, but my scores went nowhere. Later, I switched to Claude Code, which found the biggest improvement of the entire competition in about 20 minutes.

AI put this article together from my conversation logs and project documents, and the numbers come from those records. This is my post-competition review: what we did, what I learned, where I went wrong, and, finally, how I finished.

How the Competition Worked

The rules were straightforward. We could use only the three official tables covering 2016–2022—market data, 33 factors, and 10 Barra style factors—with no external data. Each day, the platform called my submitted predict function with up to 120 days of history. I scored the 300 stocks, and the official backtesting library constructed a portfolio subject to the constraints. RankIC—the rank correlation between each day’s scores and subsequent returns—accounted for 70% of the score. IR—excess return over the benchmark divided by tracking error—accounted for the other 30%.

There were public and private leaderboards, evaluated on different hidden periods. Only the final submission counted for the private leaderboard. We could submit up to three times a day, with an October 14 deadline. The organizers could also spot-check entries by retraining on the released data using train.py; if the results didn’t match, the entry would be disqualified.

A Two-Week Play-by-Play

In early October, I asked Codex to work through the competition requirements. It read all three official PDFs and wrote up its interpretation. I then had it create AGENTS.md with some ground rules: select models using walk-forward validation—train through one year, test on the next—instead of tuning to the public leaderboard. I also added a rule of my own: exclude 2022 from training and reserve it for testing. That turned out to be a mistake.

The next few days were the Codex phase. Its engineering was solid: experiment tracking, backtesting, packaging, and acceptance checks. Eventually, we accumulated nearly 200 experiment directories, 1,239 backtests, and 354 analysis documents. But the results remained disappointing. My first submission had a RankIC of 0.0495 and an IR of 0.74, placing 87th out of 104, with no substantial improvement afterward.

Usage limits were another problem. I originally alternated between two Codex accounts on ChatGPT Plus because one was nowhere near enough. Then even two proved insufficient, so I added a $100-a-month Pro subscription. It ran nonstop for a day or two without delivering any improvement, which was deeply disappointing. ChatGPT’s models were also unusually slow at the time—token throughput (tps) was a problem—which made things worse.

Out of ideas, on October 9 I decided to try Claude Code and asked it to check our setup against the original official rules. Within about 20 minutes, it found the issue: the rules explicitly allowed training on all data from 2016–2022. Excluding 2022 was just a restriction I had written into AGENTS.md myself. Once we included it, the same approach took public-leaderboard RankIC from about 0.054 to about 0.073. This was the biggest improvement of the entire competition, with no new technique—just the removal of a self-imposed restriction. All subsequent training—an ensemble of LightGBM and CatBoost—along with acceptance checks and the final submission package, was handled in Claude Code.

After adding 2022, the ensemble’s walk-forward IC plateaued at 0.0905–0.0915. In the final few days, I had Claude Code try every direction we could think of. Almost all of them either did nothing or made the overall result worse:

There were a few submission issues, too. Packages of around 52 MB triggered “judger stopped responding” three times, but reuploading the same file worked. Packages of 23 MB or less never failed. I also asked AI to build two test packages with slow-moving signals weighted at 30% and 60%, setting the decision criteria before submission. The 60% version was clearly worse—IR fell by 0.94—while 30% was flat, suggesting the original package was already at the sweet spot. One reminder to myself: since only the last submission counts for the private leaderboard, always reupload the safe package after testing. Someone in the group also suggested computing a closed-form ridge regression inside predict using returns that had already materialized within the input window. Reading the guidelines literally, AI considered this “refitting during evaluation” and recommended getting written confirmation first. The offline gain was small anyway—at most +0.0005 when blended—so we left it out.

After the competition, I first cleaned up the disk, deleting regenerable intermediate files to bring the project down from 67 GB to 26 GB. I kept the code, configurations, and predictions needed for reproducibility. Then I read the winner’s open-source code (CSI300-Challenge-2026) and compared our approaches point by point.

Results and the Gap

On the public leaderboard, I had a RankIC of 0.0773 and an IR of 2.58, placing 41st out of 153. On the private leaderboard, I had a RankIC of 0.0708 and an IR of 4.37, placing 11th out of 90. The winner’s private-leaderboard figures were 0.0825 and 5.21. Since the two leaderboards had different numbers of entrants, the rankings aren’t directly comparable. Still, three things stood out.

The gap was in RankIC, not IR. The winner’s private-leaderboard Score was 0.145 above mine. About 83% of that gap came from RankIC and only about 17% from IR, yet I had spent a lot of effort on IR tricks. Public-leaderboard IR was very noisy: mine rose from 2.58 to 4.37, while the winner’s fell from 5.57 to 5.21. Other entrants often saw swings of several points between the two leaderboards as well.

I wasn’t behind locally; I lost more ground when the evaluation period changed. In local validation, my RankIC was about 0.092 versus the winner’s 0.0887, so mine was no lower. But from the public to the private leaderboard, mine fell by about 8.4%, while the winner’s fell by only about 2.9%.

Local validation was well calibrated. Tests with training cutoffs moved back by one and two years predicted ICs of about 0.077 and 0.069. The actual public and private results were 0.0773 and 0.0708—roughly in line. Relying on this validation setup rather than the public leaderboard to select models had kept us on track.

Learning as I Went

I had almost no background in quantitative finance. My approach to learning was simple: let AI do the work, then keep asking questions. Whenever it reached a conclusion, I asked “Why?”, “Could this be overfitting?”, or “Are the public and private leaderboards similar?” It would answer with data, and I would ask another assistant to review it. That gave me a basic grasp of a few things.

How to tell whether a model is any good. Local evaluation needs walk-forward validation: train through one year and test on the next, rather than scoring on years used in training. The public leaderboard is only a cross-check because it is so noisy. According to AI’s estimates, the standard error over that short public evaluation period was roughly ±0.01 for RankIC and ±1.4 for IR. A gap of a few points could reflect skill or luck; it was hard to tell. Making lots of submissions and picking the best score also inflates the result through selection effects.

How to tell whether an improvement is real. With any gain, first ask whether the model has peeked into the future. That was how we found the two leakage issues. The next question is whether the gain holds up in a new period. The private leaderboard uses a different, later stretch of data, with no further model updates, so degradation in tests with longer train–test gaps matters more than the local score itself. Most of this understanding comes from AI’s explanations and our experimental results. I haven’t studied it systematically, so some details may still be inaccurate.

Don’t put all your eggs in one basket—or assume similar services are interchangeable. I used to think ChatGPT Plus was much like competing subscriptions, so ChatGPT alone would be enough. This experience felt very different. For Codex, the $20 Plus plan was barely usable at this intensity. Alternating between two accounts still wasn’t enough, and even after adding the $100 Pro plan and running it for a day or two, I got nowhere. With Claude Code, I used only the $20 Pro plan, yet its allowance felt close to that of the $100 Pro plan, and the experience was fast and accurate. This is just my experience from this one project; models and plans keep changing. But it taught me that when I’m stuck, I should try a few different tools rather than keep grinding away with one.

Looking Back

First, what went well: model selection relied on three layers of validation—walk-forward, leave-one-year-out, and tests with longer train–test gaps—not the public leaderboard. Every apparent improvement was checked for peeking into the test period, which caught the two leakage issues. Decision criteria for test packages were written in advance, and the final package was chosen ahead of time rather than swapped out to chase a better public score. We also reran train.py independently on the local machine, and predictions matched value for value on sampled days.

My own restrictions were stricter than the rules. The “no training on 2022” rule shaped every experiment during the Codex phase and delayed our biggest gain by several days. When we discovered it, the rule was still in AGENTS.md, so a new session would have excluded 2022 all over again. Next time, every self-imposed rule needs a rationale and conditions for removing it. Every few days, I should check those rules against the official text: use what is allowed.

The local backtest didn’t match the platform. The Codex branch still used an early proxy setup, with different holding counts, constraints, and costs from the official evaluation. Its IR wasn’t comparable to the platform’s. I only discovered this after reading the winner’s code. The winner’s local IR was about 5.2, closely matching 5.57 on the public leaderboard and 5.21 on the private one, because the local backtest copied the constraints and costs from the official example. Next time, I should replicate the evaluation in week one and avoid optimizing IR until the local and platform results line up.

I kept adding variants to exhausted approaches and focused on the wrong things. Nearly a hundred variants and over a hundred hyperparameter trials produced gains lost in the noise. IR tricks—smoothing, hysteresis bands, coarse bucketing, and neutralization—also left the combined result flat or worse. The real gap was out-of-sample RankIC degradation. The winner found gains elsewhere: freezing a factor network and then learning relative strength among neighbors with similar styles, plus computing a closed-form correction on the fly using the latest 20 labels whose outcomes were already known. Both targeted changes across periods. Next time, out-of-sample degradation should be the main metric, validation with longer train–test gaps should be routine, and each direction should get a fixed budget. After several attempts without a gain, move on.

Nobody was keeping track of rule clarifications. We shelved closed-form ridge regression, while the winner cited a clarification from the organizers and used it, gaining about +0.004 in local RankIC. I don’t know how much it was worth on the private leaderboard, but we missed an opportunity because we lacked information. Next time, check the official group and website daily, ask for verbal clarifications in writing, and test the offline gain first when something is uncertain.

There were two engineering risks as well. The 52 MB package caused three platform failures. A transient failure during private evaluation could have been especially costly, so the final package should be smaller, with a lightweight backup ready in advance. AGENTS.md had also grown to nearly 80 KB. New sessions read it first, consuming context and risking confusion from outdated rules. It should contain only rules that are still in force.

On Vibe Coding

For those two weeks, my main jobs were setting requirements, making decisions, and challenging answers: “Why aren’t you doing this?”, “What if the ranking isn’t high enough?”, “Could the top entries be overfitting?” AI did everything else.

Codex and Claude Code performed very differently in this competition. Codex’s engineering was solid, but it kept following the rules I had written without stepping back to question the exclusion of 2022. Claude Code read the official text first, checked my restrictions against it, and quickly found the problem. Of course, this was just one competition. The tools took over at different stages, with different prompts and context, so this is not a general verdict on either one. What I can say is that getting a fresh pair of eyes on the same problem and checking the original rules again was far cheaper than piling on more experiments.

Overall, AI was good at volume and discipline: running lots of experiments, auditing for leakage, writing decision criteria in advance, and checking reproducibility. It also corrected its own mistakes. It was less good at changing direction without a push, tended to tweak familiar model families, interpreted rules conservatively, and offered probabilities that were useful only as rough estimates. Choosing a direction, deciding whether to explore gray areas in the rules, knowing when to cut losses, and refusing to chase only the public leaderboard still required human judgment. Giving one assistant a dedicated review and compliance role was worthwhile, too—that was how we found the biggest gain.

The biggest improvement came from removing a self-imposed restriction. The biggest change in my understanding came after seeing the private leaderboard: the real gap was in the robustness of out-of-sample RankIC, not the IR tricks I had spent so much time on. As a newcomer, my biggest takeaway was learning to question whether a result was real: had it peeked into the future, would it hold up in a different period, and did it reflect skill or just public-leaderboard luck? Next time, I’ll align the evaluation setup first, keep a close eye on the rules, and focus on preventing performance drops across periods.

And Finally, the Result

The competition had three tracks: School of Information students, first-year students, and students from other schools. I was in the third track. I finished 11th out of 90 on the full private leaderboard and second in my track, earning a ¥200 prize. The winner of my track was only one place above me overall. I couldn’t match the School of Information students, but I did finish ahead of every first-year student on the full private leaderboard.

Honestly, I got lucky. I wasn’t near the top of the public leaderboard; I only moved up on the private one. As I mentioned, the public leaderboard was noisy, so luck played a part. What impressed me more was how powerful AI could be: someone with no quant competition experience could use it to build something competitive with the other entries and learn a lot along the way. The prize wasn’t much, but the experience meant a great deal.