This week, the Philadelphia Eagles and Kansas City Chiefs players and coaches will put in countless hours in film rooms to get ready for the Super Bowl. In an effort to identify opponent trends they can exploit and to shore up any weaknesses, they will study positions, plays, and formations.
While reducing the time and expense associated with film study for Super Bowl-bound teams (and all NFL and college football teams), new artificial intelligence (AI) technology being developed by engineers at Brigham Young University could potentially improve game strategy by using the power of big data.
BYU professor D.J. Lee, master’s student Jacob Newman and Ph.D. students Andrew Sumsion and Shad Torrie are using AI to automate the time-consuming process of analyzing and annotating game footage manually.
The researchers developed an algorithm using deep learning and computer vision that can reliably identify players from game footage and estimate the composition of the offensive squad, a task that can take a lot of the work of many video assistants.
“We were having a conversation about this and realized, whoa, we could probably teach an algorithm to do this,” said Lee, a professor of electrical and computer engineering. “So we set up a meeting with BYU Football to learn their process and immediately knew, yeah, we can do this a lot faster.”
Big data can help us know the strategies of this team, or the tendencies of that coach. It could help you know if they are likely to go for it on 4th Down and 2 or if they will punt. The idea of using AI for sports is really cool, and if we can give them even 1% of an advantage, it will be worth it.
Professor D.J. Lee
Even though the research is still in its early stages, the team has already achieved greater than 90% accuracy with its algorithm for player detection and labeling, along with 85% accuracy for identifying formations. They think that in the future, the technology will be able to do away with the time-consuming and ineffective practice of manual annotation and analysis of recorded video utilized by NFL and collegiate teams.
Lee and Newman first looked at real game footage provided by BYU’s football team. They began to evaluate it and quickly discovered they required some extra perspectives in order to train their algorithm correctly. They then manually annotated 1,000 photos and videos from Madden 2020, a game that shows the field from above and behind the offensive.
Through the usage of those photographs, they trained a deep-learning algorithm to identify the players and then fed that information into a Residual Network framework to calculate the players’ positions.
Finally, their neural network uses the location and position information to determine what formation (of more than 25 formations) the offense is using anything from the Pistol Bunch TE to the I Form H Slot Open.
Lee said the algorithm can accurately identify formations 99.5% when the player location and labeling information is correct. The I formation, where four players are lined up one in front of the next center, quarterback, fullback and running back proved to be one of the most challenging formations to identify.
Lee and Newman said the AI system could also have applications in other sports. For instance, in baseball, it may map out player locations on the field and pinpoint recurring trends to help teams improve their defense against particular batters. Alternatively, it might be used to find soccer players in order to develop more successful formations.
“Once you have this data there will be a lot more you can do with it; you can take it to the next level,” Lee said. “Big data can help us know the strategies of this team, or the tendencies of that coach. It could help you know if they are likely to go for it on 4th Down and 2 or if they will punt. The idea of using AI for sports is really cool, and if we can give them even 1% of an advantage, it will be worth it.”
