OpenAI, the nonprofit research organization founded by Elon Musk, can claim a world first: its artificial intelligence system trained to play the complex strategy game Dota 2 has defeated a champion of the world of electronic sports today. The competition was held in San Francisco and was called OpenAI Five Finals, which ended the public demonstrations of the organization of its game technology Dota on a high note.
The competition on the human side included five of the best professionals Dota 2 from the OG team, which won the most coveted electronic sports prize last year when it won first place at The International, the main annual premiere. Dota 2 tournament with prizes that now total $ 25 million. OG faced a contest of the best of three against OpenAI Five bots, all trained using the same deep reinforcement learning techniques and independently controlled by different layers of the same system. Reinforcement learning is, in effect, a trial-and-error approach to personal improvement, in which the AI is placed in the game environment without any understanding of how the game works and is trained using reward systems and other coping mechanisms. incentive.
Today's performance is by far the highest quality demonstration of OpenAI Five's capabilities to date, as the system lost almost two games to less capable electronic sports teams last August. According to Greg Brockman, OpenAI's technology director, OpenAI Five improves by playing alone in an accelerated virtual environment. "OpenAI Five is based on a deep reinforced learning, which means we do not code it to play, we codify how to learn," Brockman told the crowd before the competition. "In its 10 months of existence, it has already played 45,000 years of play Dota 2 That's a lot, it's still not boring."
Dota 2 is a very complex strategy game, involving more than 100 unique characters. , deep skill trees and item lists, and an incredible variety of variables that are reproduced on the screen at a given time in a game. As such, OpenAI imposes certain limits when its artificial intelligence system plays professional players, highlighting the number of heroes used by the two teams of five players.
In this case, each squadron had 17 heroes to choose from. OpenAI also chose the game mode called "Captain Draft", which allows each team to strategically prohibit the heroes to prevent the other team from selecting those characters before using a different selection order. That allows the captain to build strengths between combinations of heroes and take advantage of the weaknesses of enemy heroes through strong counters once the teams begin to fill the list one by one. Like the previous matches, OpenAI also disabled the invocation and illusion functions, both involve the introduction of additional variables in the form of hero copies and unique creatures that OpenAI has not trained to its system to take into account.
Beyond that, the game is played as a normal game Dota 2 with the ultimate goal of destroying the "old" enemy team or a large tower at the end of each team's territory. becomes vulnerable only when the enemy team successfully destroys smaller towers during the course of the game, between heroes fights against heroes.
In the first match of the day, OpenAI Five surprised OG and claimed the victory through a series of aggressive tactics, including the peculiar decision to spend the coin earned in the game to instantly revive the heroes at death, including at the beginning of the game. . As noted by Greg Brockman, OpenAI's technology director, OpenAI likes strategies that favor short-term gains, revealing shortcomings in mastering the long-term type of planning in which human beings excel and generally depend to win such strategy competitions. However, in this game, the anticipated repurchases paid off and OpenAI Five got an advantage that OG simply could not overcome, as the game lasted in the range of 30 minutes.
We see that this happens in test games all the time: robots buy, humans laugh and then humans lose. It is difficult to know if it will happen here too …
– Greg Brockman (@gdb) April 13, 2019
In the second game, OpenAI performed even better, gaining an early advantage against OG in the first minutes and then ruthlessly advancing on the human players until they achieved victory in little more than half the time needed to win the first game. Mike Cook, an avid Dota 2 player and spectator who specializes in mixing artificial intelligence and game design, noticed how unusually aggressive OpenAI Five started in the second game and how little OG was doing to combat their advances. Through the map. Cook noticed specifically how well OpenAI Five was able to take advantage of his specific hero selections.
This probably ended, unfortunately. OpenAI has four of the five best heroes classified by net worth. At ten minutes against the bots running OpenAI, this is really bad. #openaifive
– mike cook (@mtrc) April 13, 2019
For OpenAI, the victory here is not only a cause of celebration in itself, but a testament that approaches Reinforcement learning and its general philosophy about AI are producing milestones. The company will no longer perform public demonstrations of its AI bot, but is now working on software that will allow humans to collaborate with the OpenAI Five software in real time, play on a computer with the bots and learn from their strategies and strategies. peculiar and unprecedented. behavior.
It is possible that this software never reaches the public, although I was able to prove it by myself here at the event and, despite having world class AI Dota 2 on my computer, unfortunately crushed so much less dramatic than OG. But Sam Altman, co-founder and president of OpenAI, says that this work is evidence that collaboration with AI agents could generate huge benefits in the future.
"That's an important lesson to know how the world is going to work, train these things and make them work in parallel," Altman says. an interview with The Verge . "Collaboration is one of the most positive visions we have for the future of the world: AI works together with humans to make humans better and have more fun and more impact. "
Altman says that OpenAI will probably continue to dabble with Dota 2 and other videogame environments, mainly because they are such good test beds for AI and good benchmarking tools to measure progress. But he tells me that there probably will not be a videogame now that a system like OpenAI Five can not dominate at a level beyond human capacity For the artificial intelligence industry More broadly, the domain of video games may become obsolete, simple bets are required to prove that your system can learn quickly and act in the way necessary to face more difficult and real tasks with more significant benefits.
Ultimately, OpenAI wants to take their Dota 2 learnings and expand them to new domains outside of games and, eventually, to the real world. To that end, the organization is working on the use of reinforcement learning and other techniques to imbue robotic hands with more skilled, skilled and human movements.
"What OpenAI is trying to do is develop general artificial intelligence and share those benefits with the world and make sure it's safe," says Altman, referring to the search for a multipurpose artificial intelligence system capable of performing any task a human can "We were not here to win over video games, no matter how much fun it is, we are here to discover secrets along the AGI path".