This README serves to provide a form of informal documentation for the code in this repo
For information on how to setup the environment and the challenge structure, see TIL README
We developed some tools to help us manually submit and auto-submit to the evaluation server (Discord Watcher and RL Autorun)
NVIDIA's Nemo Parakeet model. Pretty standard setup with annoying bucketing that I spent too long trying to figure out last year in TIL-25
The Parakeet collection of models contains several different architectures and parameter sizes. Notable models include tdt-0.6b-v2, tdt-1.1b-v2, ctc-0.6b, ctc-1.1b, rnnt-0.6b and rnnt-1.1b. We found that finetuning the tdt-0.6b-v2 gave the highest accuracy of 0.991 on the qualifiers evaluation, though at the expense of minimal speed compared to the ctc and rnnt variants. Increasing the parameters from 0.6b to 1.1b did not yield improvements.
We also discovered that the ctc and rnnt models were pretrained on normalised transcript text that removed punctuation and capitalisation, while tdt-v2 was pretrained with case-sensitive transcripts with punctuation included. This difference was small but crucial, as finetuning without a consistent transcript format would likely lead to a degradation in performance, especially for the ctc and rnnt models where punctuation would be out of their tokenizers' vocabulary.
(add inference speedup tricks)
This year we tested YOLOv11, YOLO26, RTDETR (our best model from last year), RFDETR, DEIMv2, and EdgeCrafter. RFDETR would serve to be our strongest model, with some additional speed optimisations done through TurboJPEG.
Attempts were made to generate synthetic data by scraping objects from the images using SAM and backgrounds from free online APIs. Sadly, the backgrounds found did not play into the variety of backgrounds we were aiming for and in the interest of time, synthetic data was not significantly developed / utilised. (adding synthethic decreased accuracy mostly because our synthetic data did not address small/big object imbalance)
(We observe that the given training data contains a few extremely small objects (e.g. 6px tall missile that is barely distinguishable to the human eye) (Thus a sufficiently large resolution is needed. --> augmentation parameters have to change with resolution) (Furthermore, given the large drop in accuracy from around 0.97 in local validation split to 0.766 in the hidden test set for qualifiers, we deduce that the test set is likely to contain many instances of extremely small objects.)
(add inference speedup tricks) (add details on til25 data)
In terms of offense, ideas of using PGD and FGSM instantly came to mind but were quickly shot down by the uncertainty of the models being used by competitors. We then turned to more computer vision methods of noising the image, such as applying blurs, augments, etc.
Eventually, we developed the idea of an object-bank spam, utilising the already scraped object PNGs from the synthetic data development. Realising that this fit our budget constraints, we played heavily into it, only applying pixelation and grayscale to the objects detected by querying our CV model.
On considering defence, we initially were interested in training our models on adversarial data, in order to protect against perturbation-based attacks. This idea was not only ineffective but deemed redundant when we determined that such attacks were unlikely.
We developed CV-based defences, but seeing our CV score during the semifinals testing, we decided not to ship the defences to maximise score.
@gatastol help me fill this in to explain the NLP idea
Being given the evaluator model for nlp, we developed a prompt-injecting approach using UAT hotflipping to develop a master response. The final trigger is used in Cheese Manager.
We tested different lengths of triggers and found out that longer triggers scored better.
BM25 is still used for document retrieval, but the text returned is just the trigger.
We submitted just an algo for this as we ran out of time to rope in an LLM arbiter on top of the algorithm. Given the goal of just survival, we adopted a passive approach of making alliances.
Additionally, we discovered that the code written to determine if there was peace
between nations was bugged to only detect for is_active() on the diplomacy state
and that we could bypass the truce-break-attack cooldown, developing a sort of
backstabbing approach.
For AE, our initial design was a balanced algorithm that could handle both offense and defense smoothly, switching between modes using a priority-based directive system. The priority is set based on how specific a situation is, as well as its urgency:
Once we got to the evaluation, we realised that this balanced strategy was not really cutting it, likely due to the opponents not being advanced enough to be worth being so cautious about. We then came up with our second strategy: Berserker. This strategy ignores most of the defensive play in Balanced and only preserves the basic dodge instinct, instead focusing on all-out rushing opponent bases. This strategy was better at handling the evaluation, but it has a higher variance in score due to it being more susceptible to enemy counterattacks. These types of strategies are also considered unsustainable due to the nature of the game. If everyone uses a base rush strategy, each player would on average destroy one base and lose their own, resulting in a net of 0 points. Future strategies after this were mostly focused on finding a balance between these two strategies.
Our RL model is a ppo style model which reads the agent and base viewcones using CNNs as well as the other information through an MLP and flattens it all into a GRU.
In terms of training, we had several stages / attempts
In the end, our best model was obtained from behaviour cloning our algo, then league training mainly on self-play followed by pfsp + reward shaping self-play (hard + no mult) and “frozen_core” training (outlined in frozen core 1 and frozen core 1b) (at least thats what i can remember)
An additional consideration baked into the rewards shaping was that after destroying one base, all other bases are likely to be destroyed, intending on pointing the RL in the direction of PvP for gaining points.
Our final RL is no where good at all, we discovered during semifinals and finals that it likes to go in one specific corner and gets stuck there 😢
This README serves to provide a form of informal documentation for the code in this repo
For information on how to setup the environment and the challenge structure, see TIL README
We developed some tools to help us manually submit and auto-submit to the evaluation server (Discord Watcher and RL Autorun)
NVIDIA's Nemo Parakeet model. Pretty standard setup with annoying bucketing that I spent too long trying to figure out last year in TIL-25
The Parakeet collection of models contains several different architectures and parameter sizes. Notable models include tdt-0.6b-v2, tdt-1.1b-v2, ctc-0.6b, ctc-1.1b, rnnt-0.6b and rnnt-1.1b. We found that finetuning the tdt-0.6b-v2 gave the highest accuracy of 0.991 on the qualifiers evaluation, though at the expense of minimal speed compared to the ctc and rnnt variants. Increasing the parameters from 0.6b to 1.1b did not yield improvements.
We also discovered that the ctc and rnnt models were pretrained on normalised transcript text that removed punctuation and capitalisation, while tdt-v2 was pretrained with case-sensitive transcripts with punctuation included. This difference was small but crucial, as finetuning without a consistent transcript format would likely lead to a degradation in performance, especially for the ctc and rnnt models where punctuation would be out of their tokenizers' vocabulary.
(add inference speedup tricks)
This year we tested YOLOv11, YOLO26, RTDETR (our best model from last year), RFDETR, DEIMv2, and EdgeCrafter. RFDETR would serve to be our strongest model, with some additional speed optimisations done through TurboJPEG.
Attempts were made to generate synthetic data by scraping objects from the images using SAM and backgrounds from free online APIs. Sadly, the backgrounds found did not play into the variety of backgrounds we were aiming for and in the interest of time, synthetic data was not significantly developed / utilised. (adding synthethic decreased accuracy mostly because our synthetic data did not address small/big object imbalance)
(We observe that the given training data contains a few extremely small objects (e.g. 6px tall missile that is barely distinguishable to the human eye) (Thus a sufficiently large resolution is needed. --> augmentation parameters have to change with resolution) (Furthermore, given the large drop in accuracy from around 0.97 in local validation split to 0.766 in the hidden test set for qualifiers, we deduce that the test set is likely to contain many instances of extremely small objects.)
(add inference speedup tricks) (add details on til25 data)
In terms of offense, ideas of using PGD and FGSM instantly came to mind but were quickly shot down by the uncertainty of the models being used by competitors. We then turned to more computer vision methods of noising the image, such as applying blurs, augments, etc.
Eventually, we developed the idea of an object-bank spam, utilising the already scraped object PNGs from the synthetic data development. Realising that this fit our budget constraints, we played heavily into it, only applying pixelation and grayscale to the objects detected by querying our CV model.
On considering defence, we initially were interested in training our models on adversarial data, in order to protect against perturbation-based attacks. This idea was not only ineffective but deemed redundant when we determined that such attacks were unlikely.
We developed CV-based defences, but seeing our CV score during the semifinals testing, we decided not to ship the defences to maximise score.
@gatastol help me fill this in to explain the NLP idea
Being given the evaluator model for nlp, we developed a prompt-injecting approach using UAT hotflipping to develop a master response. The final trigger is used in Cheese Manager.
We tested different lengths of triggers and found out that longer triggers scored better.
BM25 is still used for document retrieval, but the text returned is just the trigger.
We submitted just an algo for this as we ran out of time to rope in an LLM arbiter on top of the algorithm. Given the goal of just survival, we adopted a passive approach of making alliances.
Additionally, we discovered that the code written to determine if there was peace
between nations was bugged to only detect for is_active() on the diplomacy state
and that we could bypass the truce-break-attack cooldown, developing a sort of
backstabbing approach.
For AE, our initial design was a balanced algorithm that could handle both offense and defense smoothly, switching between modes using a priority-based directive system. The priority is set based on how specific a situation is, as well as its urgency:
Once we got to the evaluation, we realised that this balanced strategy was not really cutting it, likely due to the opponents not being advanced enough to be worth being so cautious about. We then came up with our second strategy: Berserker. This strategy ignores most of the defensive play in Balanced and only preserves the basic dodge instinct, instead focusing on all-out rushing opponent bases. This strategy was better at handling the evaluation, but it has a higher variance in score due to it being more susceptible to enemy counterattacks. These types of strategies are also considered unsustainable due to the nature of the game. If everyone uses a base rush strategy, each player would on average destroy one base and lose their own, resulting in a net of 0 points. Future strategies after this were mostly focused on finding a balance between these two strategies.
Our RL model is a ppo style model which reads the agent and base viewcones using CNNs as well as the other information through an MLP and flattens it all into a GRU.
In terms of training, we had several stages / attempts
In the end, our best model was obtained from behaviour cloning our algo, then league training mainly on self-play followed by pfsp + reward shaping self-play (hard + no mult) and “frozen_core” training (outlined in frozen core 1 and frozen core 1b) (at least thats what i can remember)
An additional consideration baked into the rewards shaping was that after destroying one base, all other bases are likely to be destroyed, intending on pointing the RL in the direction of PvP for gaining points.
Our final RL is no where good at all, we discovered during semifinals and finals that it likes to go in one specific corner and gets stuck there 😢
