When I was a computer science freshman at my local university, one of the first courses we had was ‘Perspectives in Computer Science’. The purpose was to teach students how broad the computer science field is. The course changes every year: This year we were meant to show up and do something entirely different every week.
For instance, one week was about game theory. We were supposed to figure out some deep truth like the optimal strategy in the game rock-paper-scissors. We were grouped into pairs of two and given a Java project that connected to a server, our bots would compete in games of rock-paper-scissors. All we had to do was write a function that decided what move to make. Our bots were allowed to remember all past games against our current opponent. If we could figure out our opponent's strategy, we could predict their moves and adjust our moves according.
Initial decision functions were simple and deterministic. As a child, I remember my opponent opponent often just adding 2 to the past move, as their next move, as this was the most naively random move to make. If you caught them doing that, you just added one to that, and beat them fairly reliably. That was how some of the early bots worked, and when we figured out each other's strategies, we just added 1 again. This was fairly fruitless and it was trivial to write a bot that figured out the opponent's current addition value and added one to that.
The decision functions quickly got rather complicated, and so did the attempts at countering them. Some of the pairs simply were better programmers. There was an online scoreboard that allowed us to see how well we were doing. We got a point whenever we won a game and we lost a point whenever we lost a game. The bot games went fast, so the better pairs got up towards 8000 points during the first hour. The participants were mostly novice Java programmers, so the bots crashed often, and everyone were in a constant cycle of keeping their bot running when it was doing statistically well, and stopping their bot so it could be improved whenever it was crashing or losing a lot.
I didn't have a laptop at the time, so I shared the laptop with my teammate. It was a Mac with US keyboard layout, something I wasn't comfortable typing on, so he did most of the programming. I was, however, quite comfortable with my Nokia N900 mobile phone, it had an excellent physical QWERTY keyboard. It even had a ssh client. I must have gotten a bit bored doing backseat programming, as I began toying with my phone instead.
My first insight was that the hostname our bot connected to was the name of one of the public servers available for students and faculty, servers that all the new students recently had gotten ssh access to using our computer science login. I ssh'd into that server and took a look at the process listing to investigate the server process. It ran as one of the graduate students in charge of us that day. A quick look into that person's home directory and I quickly found the source code for the server. The source code was fairly straightforward and worked as intended, nothing in particular that could be used.
Each bot authenticated as the pair using our pair ID and a short four character password. This looked suspiciously weak, but still perhaps too much of a hassle to bruteforce. The password file was unfortunately not readable. Someone cared enough to make sure that was the case. But more interesting, among the files were the password generation code. It simply did a stock 32-bit integer hash of the pair id salted with unseeded randomness.
That got my attention. I made a copy of the code under my own user and ran the password generation code on the public list of pair id. The password for my pair was right.
I quickly re-read the server source code and realized it didn't care if the same pair had multiple bots, although winning against yourself gave a net of no points. I checked the scoreboard and my pair was only doing average, which was no surprise given that I had all of sudden become very unhelpful with my attention on my phone. We had some 3000 points and the best pair had over 8000 points. They must have developed some very successful heuristics.
I fetched a copy of the client code onto the server I had ssh'd into and modified it to authenticate as the winning team, to forfeit the game whenever the opponent was the winning team, and to always pick rock. I started the bot, it authenticates successfully, and oh no, my impostor bot isn't doing terribly well in the competition. I put my phone away and pay some attention to my teammate.
Moments after, I hear exclamations from one of the other tables. The winning pair is suddenly down to 5000 points, despite their bot reports winning the majority of games. One of them shouts how they're losing 100 points every time he F5 the page, urging his teammate to shut down the bot immediately. They entirely disbelieve how their bot suddenly perform so badly and they rush to undo recent changes. Meanwhile, their rivals are suddenly doing fairly well. Things are starting to get pretty heated, while most of us think their sudden fall from power is pretty funny. A few minutes later, they are down to several negative thousand points, and I turn my imposer bot offline due to pity and a bit of remorse.
Alas, it's a competition, and I have a secret weapon not even my teammate knows about. We're still not doing that well, so I turn my bot onto the new pair in the first place. The original victims are relieved their bot started working again, and slowly crawl up from the abyss of point debt I threw them into. The new victims realize something is going on, and they start complaining this is happening to them now. The graduate student examines their code, can't see anything obviously wrong, perhaps the are just unlucky. Perhaps one of the pairs have a really good bot now?
The speculation goes fairly rampant as to the cause of the mysterious sudden and expedient downfalls of some bots, and somehow it's accepted as a matter of fact as no satisfactory explanation presents itself. That's just how it is. I realize how we can't win the competition now, as the only pair unaffected by bad luck, towering well above the others in points, if I keep up the strategy of sabotaging anyone doing better than us. Tensions are running pretty high.
I turn my always-rock bot against our real bot, and against the bots of teams doing pretty badly, so everyone got bad luck at some point, and I vow to myself not to speak of this to anyone for at least 6 months.
The only thing I learned that day was that the winning strategy in a rock-paper-scissors championship is to make clones of your opponents that are overly fond of always picking rock.