SUBNET TRACKER · All subnets
SN15 · ORO
Every Bittensor Daily mention of SN15 ORO, newest first. Price changes are snapshots from each edition.
24h change −3.62%
- What happened
- SN15 (ORO)'s Race 6 produced unusually low scores for most agents, prompting miners to question the results. The team explained that race request text is deliberately withheld, unlike qualifying rounds where task details are visible, to prevent hardcoding and push developers toward more generalized agents. Separately, a miner reported that a submission missed the Race 7 start snapshot by 1.5 seconds despite completing its qualifying runs before the cutoff, and asked the team to manually include the version.
- Community
- Miners voiced frustration over the low Race 6 scores and the lack of visibility into race requests, with some asking the team directly to explain the hidden-task approach. Others defended the design, arguing that fresh, undisclosed race content each round is the best way to identify agents that generalize rather than overfit to past data.
- Official
- The team said race requests are withheld to prevent pattern matching and encourage versatile agent development, and said it is considering releasing sample queries in future races after observing how performance trends over the next few rounds.
24h change +0.62%
- What happened
- Miners debated whether the current champion agent's code, including a regex pattern for text handling, constitutes prohibited hard-coding, with the dispute pending a decision from the subnet owner. Separately, the team clarified rules on resubmission after elimination and on scoring for agents whose hotkey is deregistered mid-race.
- Community
- Participants split between defending the champion agent's logic as generic text processing and arguing it amounts to hard-coding, with some calling for the owner to review the case. The exchange grew tense at points, with accusations of team affiliation and calls to keep the discussion respectful.
- Official
- A community member acting in a clarifying role confirmed that similarity checks still apply to eliminated agents resubmitted under a new hotkey, and that deregistered hotkeys continue to have their agents scored but require re-registration to receive emissions.
24h change −0.77%
- What happened
- Miners flagged the top-ranked agent, identified as Leen, for exceeding the $5 per-run OpenRouter inference cost cap during races 1 and 2, with other agents including Konoha and Saffron Lantern v1 also named. The SN15 (ORO) team opened an investigation, later confirmed an audit of races 1 and 2 (Oct 22-23) found accounts that had raised the spend limit on their evaluation keys after creation. The team discarded the affected agents, banned 32 accounts covering 51 agent versions, and re-scored Race 2 with 'enjoy' as the new top agent. Separately, several miners reported repeated anti-cheating rejections with 18-hour cooldowns and said the CLI gave no specific reason for the rejections.
- Community
- Miners posted detailed cost and eval-run data to support the cost-cap allegations and asked the team for clarity on anti-cheat policy. Some community members praised the miners who had flagged the exploit, while others continued to report unresolved anti-cheating rejections and asked for help identifying the cause.
- Official
- The team acknowledged the investigation, later confirmed via a detailed post that the audit found manipulated OpenRouter spend limits, and announced that affected agents would be discarded and accounts banned, reminding miners via a pinned message that manipulated keys lead to discard and ban.
Read the full edition: SN15 (ORO) Bans 32 Accounts After Spend-Limit Manipulation Audit →
24h change −1.10%
- What happened
- SN15 (ORO) switched its ORO Bench to daily dynamic "worlds" for each race, pausing submissions and resetting all agent scores to prevent hardcoding and overfitting. Submissions were reopened shortly after the switch. Separately, several miners reported that newly submitted agents were incorrectly flagged by the anti-cheat system, and the team pushed updated static-analysis checks and reset cooldowns for the affected hotkeys.
- Community
- Many miners said the new qualifying score threshold is too high and the qualifying time window too short, repeatedly asking the team to lower the threshold or extend the window. A few participants instead described the reset as a fresh opportunity for newer miners to compete against previously dominant agents.
- Official
- The team said flagged agents found to be false positives would be pushed through for qualifying, updated static-analysis checks, and stated that qualifying thresholds would not change because all miners had the same amount of time.
24h change −1.48%
- What happened
- SN15 (ORO) miners asked the team for a specific timeline on upcoming changes to the incentive mechanism, but the team did not provide an exact date or race number. Several miners reported agents flagged by anti-cheating warnings or failures despite running previously published versions, and asked the team to investigate. One miner reported an agent stuck in the evaluation queue for four hours due to a recurring inference provider error.
- Community
- Miners repeatedly pressed for a concrete date on the incentive mechanism change, with some expressing frustration at the lack of specifics. Several miners flagged anti-cheating issues affecting agents they said were already using published code and asked for individual review via direct message.
24h change −1.45%
- What happened
- The SN15 (ORO) team announced upcoming changes to the incentive mechanism and advised miners to re-read the Agent Interface and Code Requirements documentation, warning that agents relying on non-dynamic behavior may suffer. Miners observed that race 18 reused the large majority of tasks from races 13-17 with identical request text, unlike earlier races that used new tasks each time. The team confirmed that new tasks would continue to be used until the incentive mechanism changes take effect. Separately, miners reported a sudden sharp increase in registration fees on the subnet.
- Community
- Miners asked for details on the incentive mechanism change plan, including which race it would start from and how previous races' weighted scores would be handled. Some miners disputed the team's claim that race 18 used new tasks, stating all 90 tasks were identical to prior races. Several miners also asked the team to lower the increased registration fee.
- Official
- The team said race 19 would use new tasks and that this would continue until the incentive mechanism changes are implemented, and pointed miners to a prior message with further details.
24h change −4.89%
- What happened
- SN15 (ORO) validators repeatedly hit evaluation infrastructure failures, logged as "environment_error," with the leaderboard becoming stuck or stuck in a loop across several races. Miners reported agents completing evaluation on one validator but failing or being skipped on another, leaving results missing from the final leaderboard. The team later changed validator logic between races aimed at reducing these failures, though a miner reported the same "environment_error" message again in a subsequent race.
- Community
- Miners flagged the stuck leaderboard and evaluation errors directly to the team and asked for fixes, including a case where a specific agent's full evaluation did not appear in final results. Separate exchanges in the channel covered unrelated, informal discussion about team names and membership.
- Official
- The team said it made changes to validator logic between races intended to reduce environment-error failures and increase successful evaluation runs.
24h change +1.97%
- What happened
- Miners in SN15 (ORO) reported technical issues affecting race outcomes, including an agent eliminated after passing qualifying but failing all evaluations, and multiple cases of agents discarded due to "generated evaluation infrastructure failure" errors. Separately, a miner reported their coldkey was compromised during an attempted coldkey swap, with moderators now verifying ownership while emissions continue.
- Community
- Miners affected by the evaluation errors asked the team to investigate and reinstate their agents in future races. A question about the current race champion appearing as a validator on-chain was clarified by another miner as resulting from the coldkey holding over 1000 alpha rather than indicating validator status.
- Official
- The official X account posted a link captioned "All in the open :)" without further detail.