One-Sentence Definition
A neural network is a computing system made up of interconnected nodes (neurons) organized in layers, designed to recognize patterns by adjusting numerical weights during training.
How It Works
A neural network has three types of layers. The input layer receives raw data -- pixel values for an image, token embeddings for text, or numerical features for tabular data. One or more hidden layers transform the data through weighted sums and activation functions. The output layer produces the result: a classification label, a probability score, or a generated token.
Each connection between neurons has a weight. During training, the network makes a prediction, measures how wrong it is (the loss), and then uses backpropagation to calculate how much each weight contributed to the error. An optimizer -- most commonly a variant of stochastic gradient descent -- adjusts the weights to reduce the loss. This cycle repeats over the training dataset for many epochs until the network's predictions are accurate enough.
Different architectures suit different problems. Convolutional neural networks (CNNs) excel at images because their filters capture spatial patterns. Recurrent neural networks (RNNs) process sequences step by step, which made them useful for early language models. Transformers replaced RNNs for most language and vision tasks by processing entire sequences in parallel using self-attention. Graph neural networks operate on non-grid data like social networks or molecular structures.
Why It Matters
Neural networks are the engine underneath virtually every modern AI system. The large language models you interact with (GPT-4, Claude, Gemini) are neural networks with hundreds of billions of parameters. The image classifiers in your phone's camera, the recommendation algorithms on YouTube, and the protein-folding predictions from DeepMind's AlphaFold are all neural networks.
Understanding the basics -- layers, weights, backpropagation -- gives you the vocabulary to evaluate AI claims, compare models, and understand why training costs run into the hundreds of millions of dollars for frontier systems.
Key Takeaway
A neural network is a layered system of artificial neurons that learns by adjusting connection weights, and it is the fundamental building block of deep learning and modern AI.
Part of the AI Weekly Glossary.