Deep learning

Deep learning learns useful representations by composing many layers of adjustable mathematical functions. Each layer transforms an input into a progressively more abstract representation, while training changes the network’s parameters to reduce errors on examples.¹ The basic unit, commonly called a neuron, combines inputs with learned weights and passes the result through a nonlinear activation function. A network with several such layers can represent relationships that are difficult to specify with hand-written rules.

Most deep-learning systems are trained with variants of gradient descent and backpropagation. Backpropagation efficiently calculates how much each parameter contributed to an error, allowing the optimizer to update millions or billions of parameters. Training usually requires a loss function, labeled or self-supervised data, regularization, and a separate evaluation set. The resulting model is then used for inference on new data, where it may classify, predict, retrieve, generate, or control without changing its learned parameters.

Deep learning’s modern form emerged from earlier work on artificial neural networks, statistical learning, and parallel computing. Convolutional neural networks became especially influential in image and speech processing because they exploit local structure and shared weights; recurrent networks were designed for sequences but were later supplemented or replaced in many applications by attention-based models.¹

The 2012 AlexNet result on the ImageNet image-recognition benchmark demonstrated the practical impact of deep convolutional networks trained with graphics processors and large datasets.² The Transformer architecture, introduced in 2017, used self-attention to model relationships across a sequence without recurrence and became a foundation for many large language and multimodal models.³ Other important families include autoencoders, generative adversarial networks, diffusion models, graph neural networks, and neural ordinary differential equations.

Scale has become a central source of deep-learning capability: larger models, more diverse data, and greater computation can improve performance, although gains depend on data quality, architecture, optimization, and evaluation. Self-supervised learning is particularly important because it extracts training signals from the structure of unlabeled data, such as predicting masked or subsequent portions of a sequence.

Pretraining followed by fine-tuning, instruction tuning, retrieval, or tool use allows one model to support many tasks. Deep-learning systems have achieved notable results in image recognition, speech transcription, protein-structure prediction, game playing, translation, and text generation.⁴ These capabilities do not imply general reliability: models can be brittle under distribution shift, reproduce biases in their data, generate plausible errors, or behave unpredictably when a task differs from the training environment.

Deep learning is used when complex patterns in large datasets can improve decisions, predictions, or interactions. In medicine, models assist with image interpretation, risk prediction, and clinical documentation, while regulators have established specific oversight frameworks for machine-learning-enabled medical devices. In science and engineering, neural networks help approximate expensive simulations, analyze instruments, discover candidate molecules, and control autonomous systems.

Consumer and industrial applications include search ranking, recommendation, fraud detection, speech interfaces, translation, machine vision, forecasting, and content generation. Deep learning also supports accessibility tools such as captioning and speech recognition. Deployment requires attention to privacy, security, labor effects, copyright, energy use, and the consequences of errors. Evaluation should therefore include technical accuracy as well as robustness, fairness, interpretability, human oversight, and performance in the real setting where the system will operate.⁵

Deep learning often succeeds through carefully engineered data and infrastructure rather than architecture alone. Data deduplication, filtering, labeling practices, augmentation, curriculum design, distributed training, numerical precision, and monitoring can materially affect a model’s behavior. Small changes in preprocessing or evaluation can also produce misleading comparisons between systems.

Several edge cases complicate the field. Neural networks can memorize rare or duplicated examples, leak sensitive information, exploit shortcuts that correlate with labels without representing the intended concept, and suffer catastrophic forgetting when trained sequentially on new tasks. Adversarial examples show that tiny, human-imperceptible input changes can sometimes alter predictions. Deep models are also used in low-resource settings, including on-device speech recognition and embedded sensors, where memory, latency, and energy constraints favor compression methods such as pruning, quantization, and distillation. These less visible engineering choices often determine whether a system is useful outside a laboratory benchmark.

Backpropagation

An algorithm for computing gradients through a neural network so its parameters can be updated during training.

Convolutional neural network

A neural-network architecture that uses convolutional filters to detect local and repeated patterns, especially in images and signals.

Transformer

An attention-based architecture that models relationships among elements in a sequence and supports highly parallel training.

Inference

The process of using a trained model to produce predictions, representations, or generated outputs for new inputs.