SingularAgent: Underperforming the human Atari benchmark

2/22/2021

Introduction

When DeepMind announced last year that they had developed Agent57, I was extremely impressed. Their agent was able to obtain a score greater than a human on 57 games from the Atari 2600 system. Agent57 interacted with the Arcade Learning Environment which is a framework that gives agents an interface to play Atari games. I agree that having an artificial intelligence play video games is an excellent way to test the intelligence of an agent. I decided that this would form the basis for my next milestone of SingularAgent.

SingularAgent is artificial intelligence software that makes key presses and mouse clicks on a computer. So far it is able to make nightly backups and generate a monthly financial summary for me. Today I can say that SingularAgent is able to play 56 Atari games. I defined "play" as being able to get a non-zero score and save a screenshot of its high score. SingularAgent was able to play the same games that DeepMind did with Agent57, with the single exception of Tennis (a tad difficult and extremely fast). Once I trained SingularAgent to be able to get a non-zero score, I moved onto the next game. So even if it was a low score on the first attempt, I would still move on. That's why the some of the scores are so low. I was more interested in getting a collection of general methods that would allow SingularAgent to play any Atari video game at a basic level. And I have achieved that.

Details

I used the Stella emulator so that SingularAgent would be able to play the Atari games. First I would play the game a few times in order to get a feel for how the game is played. I would make note of how to start the game, the gameplay, and the game over screen (if there even is one). For some of the more complicated games, I looked at the manuals on AtariAge website. After some analysis, I would code my strategy for the particular game into SingularAgent by typing commands into the console application. Once the game started, SingularAgent would make the following types of key presses:

random
scripted (keys pressed in a specific order)
dynamic (keys pressed based upon the changing pixel colors on the screen)

For each game I needed to determine whether or not to have SingularAgent look for a game over screen or just take screenshots periodically during the game to record the high score. If SingularAgent could determine that the game over screen was displayed, then it would save a screenshot and then exit Stella.

Differences

SingularAgent wasn't able to achieve a superhuman level of scores. Even so, my software is built in a way that offers some distinct advantages.

Explainable

SingularAgent belongs to the symbolic branch of artificial intelligence. No machine learning was used during this training. Therefore, every little action that SingularAgent took was explainable and logged to a file. There is no mystery as to why SingularAgent did what it did.

Reusable

SingularAgent could be described as a program that has a gigantic collection of methods that are connected and pass information along to each other. The methods are very general and reusable since they only do one small thing before passing along data to the next method. Methods used by one Atari game were reused by other Atari games which made learning a new game quicker and easier. The order of the methods and the parameters of the methods can be altered in real time, on the fly.

Efficient

Since SingularAgent didn't use machine learning, it is not computationally expensive to run. It doesn't take weeks/months of training using the GPU and CPU to spit out a useful program. For some of the simpler games, I was able to train SingularAgent to get a non-zero score in an hour or two.

Conclusion

I learned a lot training SingularAgent to play 56 games and I am excited to move on to my next milestone. Methods coded as the result of this process can be used in the future for any program that can run on a computer, not just video games (websites, applications, and etc.). I encourage everyone interested in playing the Atari games to head over to Free 80s Arcade website. You can play the Atari 2600 games in your browser (desktop or mobile compatible). See if you can beat SingularAgent's scores. Have fun!