AI Trends in 2022

AI is advancing at full pace with groundbreaking changes appearing every few weeks. The field is advancing so alarmingly that many practitioners are calling for ethical AI controls to be in place to govern the creation of AI. In many ways current AI techniques surpass our wildest dreams in their ability to create and innovate. Yet limitations still exist. Although advances toward “General AI” are being made, human-level AI is probably another decade from reality. But a decade is not a long time. Imagine a reality in which machines can do anything that people do.

The field of AI has seen exciting changes over the last few years with the discovery that pre-trained large language models are effective at solving complex AI problems. While standard research continues with incremental advances in various areas, most of the attention and press has been devoted to major advancements in language understanding, image understanding, language generation, and image generation. Also, of note is that language understanding and generation for non-natural languages, like programming languages, has also seen major advances due to large language model developments.

Most people tracking AI over the years are familiar with the rise of machine learning and search, the two branches of research that divide AI. While Machine Learning (ML) covers the fields of artificial neural nets, reinforcement learning and other forms of statistical learning, search covers automated reasoning, game playing, logic programming, constraint reasoning, theorem proving, and other symbolic manipulation concepts. Advances in machine learning have dominated news recently with the development of generative models that can make up anything that requires intelligence. This includes language, poetry, art, music, software and much more.

I’ve curated some of the more interesting developments in AI.

Transformer Based Large Language Models

At one point in time, full language comprehension was thought to require vast databases of carefully curated knowledge. Large language models do not require this type of effort. The architecture and size of LLMs are powerful enough to allow training alone to be used on a vast database of books, articles, websites and more. GPT-3 from OpenAI was the first large language model to significantly alter the landscape of language-based AI. It is a huge system with over 175 Billion parameters making it one of the largest models ever deployed commercially.

Transformers rely on completely new AI concept called “Attention”. The concept of attention Is based on the idea that all inputs need to be compared directly with all other inputs in one operation. This idea works extremely well and is computationally very efficient on modern hardware such as graphical processors which are designed for highly parallel computation. Multiple layers of attention for highly abstracted version of input which are then decoded back into full sentence or into images.

The impact of transformers on language and image processing has been dramatic. A GPT-3 generated paragraph or article is so good, it is usually indistinguishable from human writing. While not fully “General AI”, GPT-3 seems like its getting closer to that point. 

Low code and No-Code Intelligence

One of the applications of GPT-3 is code understanding and code generation. OpenAI has created a version of GPT-3 that they refer to as “Codex”. Codex is now integrated into 70 different platforms and is used for code generation and code completion.

Image Generation

A few years ago, the concept of Generative Adversarial Networks was introduced, and it was ground-breaking. The concept incorporated one neural network to generate images and another one to critique the images. The generated images were incredibly realistic. You can check out examples on “Thispersondoesnotexist.com”. The site generates faces that are so realistic it is impossible to tell that they are made by an AI system. Since then many variants of GAN have emerged to generate any type of image you can imagine.

More recently OpenAI released DALLE 2, a transformer-based image generator that combines the power of language processing with the power of image processing. A user can provide any phrase describing a scene, whether it is realistic or fantasy, and DALLE 2 will build a full image publication ready. DALLE 2 generates incredibly interesting, one-of-a-kind artwork in response to any user query.

Hybrid AI and General AI

With the advent of almost human AI the limitations of neural network based AI systems are starting to emerge. GPT-3 for example can understand many concepts, but is unable to do math, process dates, or think deeply about anything that requires a systematic and logical thought process. The thinking of GPT-3 is intelligent but limited in depth.

The solution to neural net limitations is to integrate logic processing within lower layers of the network. This mechanism is referred to as Neurosymbolic processing and it will have a major impact on neural nets in the coming years. Neurosymbolic processing will enable complete generalized AI to emerge that can both process language and images as well as think logically.

Game Play

Game playing systems are one of the more interesting applications of AI to emerge in recent years. The ancient game of GO was once thought to be impossible for computers to beat humans. A team of AI scientist put Reinforcement Learning together with Neural Networks and a game playing algorithm to build a hybrid system that can win any GO game against any human. Since that time DeepMind has built game playing systems for all the major games. Related techniques have even been applied to poker; another game that was once thought impossible to beat humans.

Folding Proteins

Folding proteins is a practical application, beyond games, for the Deepmind GO method. The GO team realized that the process of finding viable protein folding configurations was similar to the search techniques used in game playing. Protein folding using Deepmind and other related models has revolutionized the field of biology and medicine, providing accurate predictions of the three-dimensional structures of proteins.

The ability to accurately predict the three-dimensional structure of a protein from its amino acid sequence is of great importance in biology and medicine. Proteins are the molecules that carry out the majority of the biochemical functions in cells, and their three-dimensional structures are crucial for their function.

Many diseases are caused by proteins with abnormal structures and understanding the three-dimensional structure of proteins is essential for designing drugs that target specific proteins. Traditionally, the three-dimensional structures of proteins have been determined experimentally, using techniques such as x-ray crystallography and nuclear magnetic resonance spectroscopy. However, these techniques are expensive and time-consuming, and they can only be used to determine the structures of relatively small proteins. In recent years, computational methods have been developed that can predict the three-dimensional structures of proteins from their amino acid sequences. These methods are much faster and less expensive than experimental methods, and they can be used to predict the structures of very large proteins.

Autonomous Vehicles

AI is the essential technology enabling autonomous vehicles to drive themselves. In the US 38 states have enacted laws to enable autonomous vehicles. In China, robotaxis are now in regular use in many cities. AI is used in autonomous vehicles to achieve situational awareness, the ability to understand one’s surroundings, and navigate and drive the vehicle accordingly. AI is also used to detect pedestrians, cyclists, and obstacles in the vehicle’s path. AI is used in autonomous vehicles to make decisions about when to change lanes, turn, and brake. In the future, AI will be used to enable cars to communicate with each other and with infrastructure such as traffic lights and road signs. This will allow cars to avoid accidents and improve traffic flow.

Accelerated AI Hardware

The rapid advancement in the applications of AI is, in part, due to the development of efficient algorithms that implement AI on specialized hardware. Nvidia, for example, is the world leader in Graphical Processing Unit hardware. Large neural nets run at accelerated rates on Nvidia cards. Training Deep learning nets is very expensive and time consuming. Accelerated neural networks can train at 10000s of times the speed of standard CPU based processes.

The Future

AI is one of the most important technologies of our time and will affect nearly every aspect of our lives in the coming years. Machines that can think like people, that can solve problems and replace human efforts will revolutionize business, commerce, medicine, engineering, education, and everything else. While current AI systems are limited to single purpose implementations, emerging “General” AI systems will soon integrate multiple forms of intelligence processing into one hybrid system that can think like a person.