Yann Lecun
(153 articles)
Convolution
Mathematical operation used in signal processing and image processing to combine two functions, resulting in a third function that represents how one function modifies the other.
Generality: 870
Tensor
Multi-dimensional array used in mathematics and computer science, serving as a fundamental data structure in neural networks for representing data and parameters.
Generality: 920
Parameterized
Model or function in AI that utilizes parameters to make predictions or decisions.
Generality: 796
Loss Optimization
Process of adjusting a model's parameters to minimize the difference between the predicted outputs and the actual outputs, measured by a loss function.
Generality: 886
Quantization
Process of reducing the precision of the weights and activations in neural network models to decrease their memory and computational requirements.
Generality: 673
ANN
Artificial Neural Networks
Artificial Neural Networks
Computing systems inspired by the biological neural networks that constitute animal brains, designed to progressively improve their performance on tasks by considering examples.
Generality: 875
Neural Network
Computing system designed to simulate the way human brains analyze and process information, using a network of interconnected nodes that work together to solve specific problems.
Generality: 932
Connectionist AI
Set of computational models in AI that simulate the human brain's network of neurons to process information and learn from data.
Generality: 900
Next Word Prediction
Enables language models to predict the most probable subsequent word in a text sequence using generative AI techniques.
Generality: 780
Natural Language Problem
Challenges encountered in understanding, processing, or generating human language using computational methods.
Generality: 875
Generalization
Ability of a ML model to perform well on new, unseen data that was not included in the training set.
Generality: 891
Motor Learning
Process by which robots or AI systems acquire, refine, and optimize motor skills through experience and practice.
Generality: 675
Supervision
Use of labeled data to train ML models, guiding the learning process by providing input-output pairs.
Generality: 890
Training
Process of teaching a ML model to make accurate predictions or decisions, by adjusting its parameters based on data.
Generality: 940
Human-Level AI
AI systems that can perform any intellectual task with the same proficiency as a human being.
Generality: 945
AGI
Artificial General Intelligence
Artificial General Intelligence
AI capable of understanding, learning, and applying knowledge across a wide range of tasks, matching or surpassing human intelligence.
Generality: 905
Unsupervised Learning
Type of ML where algorithms learn patterns from untagged data, without any guidance on what outcomes to predict.
Generality: 905
Artificial Neuron
Computational models inspired by biological neurons, serving as the foundational units of artificial neural networks to process input and output signals.
Generality: 825
MCP neuron
Early computational model of a biological neuron forming the basis for artificial neural networks.
Generality: 500
ML
Machine Learning
Machine Learning
Development of algorithms and statistical models that enable computers to perform tasks without being explicitly programmed for each one.
Generality: 965
Autonomous Agents
Systems capable of independent action in dynamic, unpredictable environments to achieve designated objectives.
Generality: 875
Supervised Learning
ML approach where models are trained on labeled data to predict outcomes or classify data into categories.
Generality: 882
Machine Understanding
Capability of AI systems to interpret and comprehend data, text, images, or situations in a manner akin to human understanding.
Generality: 828
Image Recognition
Ability of AI to identify objects, places, people, writing, and actions in images.
Generality: 854
Linear Separability
The ability of a dataset to be perfectly separated into two classes using a straight line in two dimensions or a hyperplane in higher dimensions.
Generality: 500
Pattern Recognition
The identification and classification of patterns in data using computational algorithms, essential for enabling machines to interpret, learn from, and make decisions based on complex datasets.
Generality: 825
Feed Forward
Essential structure of an artificial neural network that directs data or information from the input layer towards the output layer without looping back.
Generality: 860
Function Approximation
Method used in AI to estimate complex functions using simpler, computationally efficient models.
Generality: 810
Invariance
Property of a model or algorithm that ensures its output remains unchanged when specific transformations are applied to the input data.
Generality: 830
Inference
Process by which a trained neural network applies learned patterns to new, unseen data to make predictions or decisions.
Generality: 861
Chatbot
Software application designed to simulate conversation with human users, often over the Internet.
Generality: 805
Regularization
Technique used in machine learning to reduce model overfitting by adding a penalty to the loss function based on the complexity of the model.
Generality: 845
Overfitting
When a ML model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data.
Generality: 890
CNN
Convolutional Neural Network
Convolutional Neural Network
Deep learning algorithm that can capture spatial hierarchies in data, particularly useful for image and video recognition tasks.
Generality: 916
Local Weight Sharing
Technique where the same weights are used across different positions in an input, enhancing the network's ability to recognize patterns irrespective of their spatial location.
Generality: 690
Statistical AI
Utilizes statistical methods to analyze data and make probabilistic inferences, aimed at emulating aspects of human intelligence through quantitative models.
Generality: 890
Speech-to-Text Model
A computational model designed to convert spoken language into written text using AI and linguistic pattern recognition.
Generality: 805
Universal Learning Algorithms
Theoretical frameworks aimed at creating systems capable of learning any task to human-level competency, leveraging principles that could allow for generalization across diverse domains.
Generality: 840
Learnability
Capacity of an algorithm or model to effectively learn from data, often measured by how well it can generalize from training data to unseen data.
Generality: 847
Dualism
Theory or concept that emphasizes the division between symbolic (classical) AI and sub-symbolic (connectionist) AI.
Generality: 830
EBM
Energy-Based Model
Energy-Based Model
Class of deep learning models that learn to associate lower energy levels with more probable configurations of the input data.
Generality: 625
Saturating Non-Linearities
Activation functions in neural networks that reach a point where their output changes very little, or not at all, in response to large input values.
Generality: 575
MLP
Multilayer Perceptron
Multilayer Perceptron
Type of artificial neural network comprised of multiple layers of neurons, with each layer fully connected to the next, commonly used for tasks involving classification and regression.
Generality: 775
DL
Deep Learning
Deep Learning
Subset of machine learning that involves neural networks with many layers, enabling the modeling of complex patterns in data.
Generality: 905
Backpropagation
Algorithm used for training artificial neural networks, crucial for optimizing the weights to minimize error between predicted and actual outcomes.
Generality: 890
DNN
Deep Neural Networks
Deep Neural Networks
Advanced neural network architectures with multiple layers that enable complex pattern recognition and learning from large amounts of data.
Generality: 916
Subsymbolic AI
AI approaches that do not use explicit symbolic representation of knowledge but instead rely on distributed, often neural network-based methods to process and learn from data.
Generality: 900
Hidden Layer
Layer of neurons in an artificial neural network that processes inputs from the previous layer, transforming the data before passing it on to the next layer, without direct exposure to the input or output data.
Generality: 861
Forward Propagation
Process in a neural network where input data is passed through layers of the network to generate output.
Generality: 830
Feature Extraction
Process of transforming raw data into a set of features that are more meaningful and informative for a specific task, such as classification or prediction.
Generality: 880
Function Approximator
Computational model used to estimate a target function that is generally complex or unknown, often applied in machine learning and control systems.
Generality: 806
Weight Initialization
An essential process in neural network training that involves setting the initial values of the model's weights to influence learning effectiveness and convergence.
Generality: 675
Node
A fundamental unit within a neural network or graph that processes inputs to produce outputs, often reflecting the biological concept of neurons.
Generality: 500
Batch
A collection of data samples processed simultaneously in a single step of a neural network's training process.
Generality: 500
Weight Decay
Regularization technique used in training neural networks to prevent overfitting by penalizing large weights.
Generality: 730
Autoencoder
Type of artificial neural network used to learn efficient codings of unlabeled data, typically for the purpose of dimensionality reduction or feature learning.
Generality: 815
Max Pooling
Downsampling technique that reduces the dimensionality of input data by selecting the maximum value from a specified subset of the data.
Generality: 695
Autonomous Learning
Systems capable of learning and adapting their strategies or knowledge without human intervention, based on their interactions with the environment.
Generality: 870
Speech-to-Speech Model
Systems that directly convert spoken language into another language through AI, enabling real-time translation and cross-lingual communication.
Generality: 809
Artificial Curiosity
Algorithmic mechanism in AI that motivates the system's behavior to learn inquisitively and explore unfamiliar environments.
Generality: 625
Computational Creativity
The study and building of software and algorithms that exhibit behaviors deemed creative in humans, such as generating original artwork, music, or solving problems in unique ways.
Generality: 710
SotA
State of the Art
State of the Art
The highest level of performance achieved in a specific field, particularly in AI, where it denotes the most advanced model or algorithm.
Generality: 720
Meta-Learning
Learning to learn involves techniques that enable AI models to learn how to adapt quickly to new tasks with minimal data.
Generality: 858
Catastrophic Forgetting
Phenomenon where a neural network forgets previously learned information upon learning new data.
Generality: 686
Siamese Network
Type of neural network architecture that involves two or more identical subnetworks sharing the same parameters and weights, typically used for tasks like similarity learning and verification.
Generality: 600
Wake Sleep
Biologically inspired algorithm used within unsupervised learning to train deep belief networks.
Generality: 540
Transfer Learning
ML method where a model developed for a task is reused as the starting point for a model on a second task, leveraging the knowledge gained from the first task to improve performance on the second.
Generality: 870
Continuous Learning
Systems and models that learn incrementally from a stream of data, updating their knowledge without forgetting previous information.
Generality: 870
Early Stopping
A regularization technique used to prevent overfitting in ML models by halting training when performance on a validation set begins to degrade.
Generality: 675
AI Safety
Field of research aimed at ensuring AI technologies are beneficial and do not pose harm to humanity.
Generality: 870
Object Detection
Computer vision technique that identifies and locates objects within an image or video frame.
Generality: 790
Classifier
ML model that categorizes data into predefined classes.
Generality: 861
Narrow AI
Also known as Weak AI, refers to AI systems designed to perform a specific task or a narrow range of tasks with a high level of proficiency.
Generality: 760
DBN
Deep Belief Network
Deep Belief Network
A type of artificial neural network that is deeply structured with multiple layers of latent variables, or hidden units.
Generality: 851
One-Shot Learning
ML technique where a model learns information about object categories from a single training example.
Generality: 542
Model Compression
Techniques designed to reduce the size of a machine learning model without significantly sacrificing its accuracy.
Generality: 715
Feature Learning
Automatically learning representations or features from raw input data in order to improve model performance and reduce dependency on manual feature engineering.
Generality: 500
Denoising Autoencoder
A neural network designed to reconstruct a clean input from a corrupted version, enhancing feature extraction by learning robust data representations.
Generality: 806
Sparse Autoencoder
Type of neural network designed to learn efficient data representations by enforcing sparsity on the hidden layer activations.
Generality: 625
ZSL
Zero-Shot Learning
Zero-Shot Learning
ML technique where a model learns to recognize objects, tasks, or concepts it has never seen during training.
Generality: 660
Initialization
Process of setting the initial values of the parameters (weights and biases) of a model before training begins.
Generality: 865
Data Efficient Learning
ML approach that requires fewer data to train a functional model.
Generality: 791
Similarity Learning
A technique in AI focusing on training models to measure task-related similarity between data points.
Generality: 675
Landmarks
Key points in an image used as reference for computer vision and AI systems to understand and manipulate visual data.
Generality: 500
Pretrained Model
ML model that has been previously trained on a large dataset and can be fine-tuned or used as is for similar tasks or applications.
Generality: 860
Latent Space
Abstract, multi-dimensional representation of data where similar items are mapped close together, commonly used in ML and AI models.
Generality: 805
Embedding Space
Mathematical representation where high-dimensional vectors of data points, such as text, images, or other complex data types, are transformed into a lower-dimensional space that captures their essential properties.
Generality: 700
Embedding
Representations of items, like words, sentences, or objects, in a continuous vector space, facilitating their quantitative comparison and manipulation by AI models.
Generality: 865
Straight-Through Estimator
Technique used in training neural networks to enable the backpropagation of gradients through non-differentiable functions or operations.
Generality: 555
Gradient Clipping
A technique used to mitigate the exploding gradient problem during the training of neural networks by capping gradients to a specified value range.
Generality: 625
Model Layer
Discrete level in a neural network where specific computations or transformations are applied to the input data, progressively abstracting and refining the information as it moves through the network.
Generality: 805
End-to-End Learning
ML approach where a system is trained to directly map input data to the desired output, minimizing the need for manual feature engineering.
Generality: 800
Image-to-Text Model
AI systems that convert visual information from images into descriptive textual representations, enabling machines to understand and communicate the content of images.
Generality: 755
Sequence Model
Model designed to process and predict sequences of data, such as time series, text, or biological sequences.
Generality: 830
Sequence Prediction
Involves forecasting the next item(s) in a sequence based on the observed pattern of prior sequences.
Generality: 825
Generative AI
Subset of AI technologies that can generate new content, ranging from text and images to music and code, based on learned patterns and data.
Generality: 830
Discriminative AI
Algorithms that learn the boundary between classes of data, focusing on distinguishing between different outputs given an input.
Generality: 840
Recognition Model
Element of AI that identifies patterns and features in data through learning processes.
Generality: 790
Autoregressive Sequence Generator
A predictive model harnessed in AI tasks, particularly involving times series, which leverages its own prior outputs as inputs in subsequent predictions.
Generality: 650
Sequential Models
Type of data models in AI where the arrangement of data points or events adhere to a specific order for predictive analysis and pattern recognition.
Generality: 815
Convergence
The point at which an algorithm or learning process stabilizes, reaching a state where further iterations or data input do not significantly alter its outcome.
Generality: 845
Text-to-Image Model
Converts descriptive text inputs into visual images through AI-generated interpretations.
Generality: 655
Overparameterized
ML model that has more parameters than the number of data points available for training.
Generality: 555
Loss Landscape
The topographical representation of a neural network's loss function showcasing the variations in loss values across different parameter settings.
Generality: 500
Teacher Model
Pre-trained, high-performing model that guides the training of a simpler, student model, often in the context of knowledge distillation.
Generality: 561
Robustness
Ability of an algorithm or model to deliver consistent and accurate results under varying operating conditions and input perturbations.
Generality: 885
Activation Data
Intermediate outputs produced by neurons in a neural network when processing input data, which are used to evaluate and update the network during training.
Generality: 575
GLU
Gated Linear Unit
Gated Linear Unit
Neural network component that uses a gating mechanism to control information flow, improving model efficiency and performance.
Generality: 665
Black Box Problem
The difficulty in understanding and interpreting how an AI system, particularly ML models, makes decisions.
Generality: 850
Synthetic Data Generation
Creating artificial data programmatically, often used to train ML models where real data is scarce, sensitive, or biased.
Generality: 795
Sample Efficiency
Ability of a ML model to achieve high performance with a relatively small number of training samples.
Generality: 815
Few Shot
ML technique designed to recognize patterns and make predictions based on a very limited amount of training data.
Generality: 675
Federated Learning
ML approach enabling models to be trained across multiple decentralized devices or servers holding local data samples, without exchanging them.
Generality: 805
Embodied AI
Integration of AI into physical entities, enabling these systems to interact with the real world through sensory inputs and actions.
Generality: 780
XAI
Explainable AI
Explainable AI
AI systems designed to provide insights into their behavior and decisions, making them transparent and understandable to humans.
Generality: 775
Inference Acceleration
Methods and hardware optimizations employed to increase the speed and efficiency of the inference process in machine learning models, particularly neural networks.
Generality: 775
Multimodal
AI systems or models that can process and understand information from multiple modalities, such as text, images, and sound.
Generality: 837
Responsible AI
Application of AI in a manner that is transparent, unbiased, and respects user privacy and value.
Generality: 815
Federated Training
Decentralized machine learning approach where multiple devices or nodes collaboratively train a shared model while keeping their data localized, rather than aggregating it centrally.
Generality: 805
Expressive Hidden States
internal representations within a neural network that effectively capture and encode complex patterns and dependencies in the input data.
Generality: 695
Deepfakes
Synthetic media produced by AI technologies that superimpose existing images or videos onto source images or videos to create realistic likenesses.
Generality: 560
Neurosymbolic AI
Integration of neural networks with symbolic AI to create systems that can both understand and manipulate symbols in a manner similar to human cognitive processes.
Generality: 675
Out of Distribution
Data that differs significantly from the training data used to train a machine learning model, leading to unreliable or inaccurate predictions.
Generality: 675
Point-wise Feedforward Network
Neural network layer that applies a series of linear and non-linear transformations to each position (or
Generality: 625
Hybrid AI
Combines symbolic AI (rule-based systems) and sub-symbolic AI (machine learning) approaches to leverage the strengths of both for more versatile and explainable AI systems.
Generality: 820
Masking
Technique used in NLP models to prevent future input tokens from influencing the prediction of current tokens.
Generality: 639
Ablation
Method where components of a neural network are systematically removed or altered to study their impact on the model's performance.
Generality: 650
Zero-shot Capability
The ability of AI models to perform tasks or make predictions on new types of data that they have not encountered during training, without needing any example-specific fine-tuning.
Generality: 775
SSL
Self-Supervised Learning
Self-Supervised Learning
Type of ML where the system learns to predict part of its input from other parts, using its own data structure as supervision.
Generality: 815
LLM
Large Language Model
Large Language Model
Advanced AI systems trained on extensive datasets to understand, generate, and interpret human language.
Generality: 827
Next Token Prediction
Technique used in language modeling where the model predicts the following token based on the previous ones.
Generality: 735
Causal AI
A form of AI that reasons using cause and effect logic to provide interpretable predictions and decisions.
Generality: 813
Base Model
Pre-trained AI model that serves as a starting point for further training or adaptation on specific tasks or datasets.
Generality: 790
Contrastive Learning
ML technique used primarily in unsupervised learning that improves model performance by teaching the model to distinguish between similar and dissimilar data points.
Generality: 720
DLMs
Deep Language Models
Deep Language Models
Advanced ML models designed to understand, generate, and translate human language by leveraging DL techniques.
Generality: 874
Self-Supervised Pretraining
ML approach where a model learns to predict parts of the input data from other parts without requiring labeled data, which is then fine-tuned on downstream tasks.
Generality: 725
Adapter Layer
Neural network layer used to enable transfer learning by adding small, trainable modules to a pre-trained model, allowing it to adapt to new tasks with minimal additional training.
Generality: 625
Continual Pre-Training
Process of incrementally training a pre-trained ML model on new data or tasks to update its knowledge without forgetting previously learned information.
Generality: 670
Post-Training
Techniques and adjustments applied to neural networks after their initial training phase to enhance performance, efficiency, or adaptability to new data or tasks.
Generality: 650
AMI
Advanced Machine Intelligence
Advanced Machine Intelligence
Refers to high-level AI systems possessing the capability to perform complex cognitive tasks with or without human-like reasoning.
Generality: 873
Scaling Laws
Mathematical relationships that describe how the performance of machine learning models, particularly deep learning models, improves as their size, the amount of data, or computational resources increases.
Generality: 835
1-N Systems
Architectures where one input or controller manages multiple outputs or agents, applicable in fields like neural networks and robotics.
Generality: 790
Scaling Hypothesis
Enlarging model size, data, and computational resources can consistently improve task performance up to very large scales.
Generality: 765
Parametric Knowledge
Information and patterns encoded within the parameters of a machine learning model, which are learned during the training process.
Generality: 849
Instruction Following Model
AI system designed to execute tasks based on specific commands or instructions provided by users.
Generality: 640
Instruction-Following
Ability to accurately understand and execute tasks based on given directives.
Generality: 725
Transformative AI
AI systems capable of bringing about profound, large-scale changes in society, potentially altering the economy, governance, and even human life itself.
Generality: 825
Self-Correction
An AI system's ability to recognize and rectify its own mistakes or errors without external intervention.
Generality: 815
VLM
Visual Language Model
Visual Language Model
AI models designed to interpret and generate content by integrating visual and textual information, enabling them to perform tasks like image captioning, visual question answering, and more.
Generality: 621
Foundation Model
Type of large-scale pre-trained model that can be adapted to a wide range of tasks without needing to be trained from scratch each time.
Generality: 835
MLLMs
Multimodal Large Language Models
Multimodal Large Language Models
Advanced AI systems capable of understanding and generating information across different forms of data, such as text, images, and audio.
Generality: 625
Agentic AI Systems
Advanced AI capable of making decisions and taking actions autonomously to achieve specific goals, embodying characteristics of agency and decision-making usually associated with humans or animals.
Generality: 775
LVLMs
Large Vision Language Models
Large Vision Language Models
Advanced AI systems designed to integrate and interpret both visual and textual data, enabling more sophisticated understanding and generation based on both modalities.
Generality: 675