Artificial intelligence AlphaStar reaches Grandmaster level in StarCraft II

The gaming artificial intelligence of the latest version of AlphaStar has opened up a new level of capabilities of intelligent systems. For the first time in history, AI has won a regular, not a modified version of the online game StarCraft II, fighting humans on conditions in which live esports players compete. And he consistently achieved the highest rank of "Grandmaster" in battles for each of the three races: Terran, Protoss and Zerg.

The turning point in nurturing gaming AI came last year when AlphaZero beat out its predecessor, AlphaGo. In 2016, she became the first AI to defeat the world champion in the game of go, Lee Sedol, but AlphaGo acted in the old-fashioned way, based on the set rules and strategies. The new AI AlphaZero is already entirely built on the principles of machine learning, learns from its own experience and creates winning strategies on its own. Even if initially he knows nothing about the game and his opponent.

The next challenge was games with incomplete information, like poker, and multiplayer strategies. They are especially difficult for AI: you need to control many of your own and enemy units, plan battles at least a few minutes in advance, develop your bases, and be able to combine different types of activities with instant switching between tasks. AlphaZero, after a series of experiments, learned to cope with this task, winning 95% of the time. But a new problem has arisen - the AI ​​does not understand the game itself, it acts by distancing itself from the process, which is why it is not able to truly evolve.

AlphaStar (zerg, red) deflects an early protoss attack

The solution came with the creation of the AlphaStar system, which learned to "create difficulties" for itself. Here, too, everything is built on machine learning, but if earlier all the agents of artificial intelligence had the goal of developing a winning strategy, now the AI ​​has deliberately begun to divide itself into "student" and "testers". During training, the latter try to reveal as many vulnerable spots of the enemy as possible, in addition, practicing dastardly techniques, such as collusion when playing in the "every man for himself" mode, giveaway games, etc. Having pumped in this way, AlphaStar challenged the strongest players in StarCraft, Dario "TLO" Wunsch and Grzegorz "MaNa" Kominets, and defeated both.

The interface was the last stumbling block. People play with limited speed of button clicks, mouse clicks, scrolling and battlefield views, they see and evaluate sets of pixels, not ready-made data packets. It took more than half a year to create a suitable control mechanism for AlphaStar, after which the AI ​​was released to the Internet, where it fought human players in, on real maps and in the same conditions. And he reached the level of 99, 8% of victories, having received the status of a grandmaster in all playable races.

AlphaStar (zerg, green) wins the final battle with late high tech units

The main achievement here is not at all in victory - it is only convincing proof that universal machine learning algorithms of this type can be used to solve complex real problems. For example - creating a drone, personal digital assistant, military or research robot that can operate without human assistance.