Interactive course · M1-T1-MLC2

Deep Learning

Artificial Neural Networks

Author: Mária HartyányiiTStudy Hungary Training and Research Centre
Reviewer: Zsolt LengyeliTStudy Hungary Training and Research Centre

A dog's image is processed through a network resembling biological neurons, then through artificial neurons; the output is Dog. — A mathematical model inspired by natural neural networks.

Learning objective

Responsible use of artificial intelligence requires an understanding of the mechanisms that govern how it works. This course explains the operating principles of advanced AI systems without requiring deep mathematical or programming knowledge.

By the end of this course, you will be able to

explain the concepts of artificial neural networks, deep learning, training data and parameters;
describe how a neural network processes inputs into outputs and learns from errors;
distinguish between models in terms of performance and resource requirements;
explain models' limitations and check their answers thoughtfully.

The brain provided the inspiration

The development of effective machine learning methods was also inspired by the study of processes in the human brain. Neuroscientists observed that connections between neurons change during learning: some connections strengthen while others weaken.

This insight led to the development of artificial neural networks, known as Artificial Neural Networks, or ANN for short.

From machine learning to deep learning

Artificial intelligence (AI): a broad concept encompassing computer-based solutions that model capabilities associated with human intelligence.

Machine learning (ML): a subfield of AI. The system recognizes patterns from data rather than relying on every rule being specified manually in advance.

Deep learning (DL): a narrower subfield of machine learning based on artificial neural networks with multiple processing layers.

Every deep learning system is both a machine learning and an AI solution. Not every machine learning method uses a neural network.

Nested fields: machine learning within artificial intelligence, and deep learning within machine learning. — AI = artificial intelligence · ML = machine learning · DL = deep learning.

The perceptron: an early milestone

One of the early milestones in the history of neural networks was the perceptron. Frank Rosenblatt, a researcher at Cornell University, presented his learning-capable system in the late 1950s; he published the perceptron model in 1958.

The Mark I Perceptron was an electromechanical device. Its 20 × 20 photocell array captured the input image at 400 sensing points. Electromechanical devices stored and adjusted the strengths of its connections.

Under controlled conditions, the system learned to distinguish simple visual patterns. It used supervised learning: the operator supplied the correct category for the input, and the perceptron compared it with its own answer. If the answer was wrong, it adjusted the connection weights.

Frank Rosenblatt beside the Mark I Perceptron's sensor panel and wiring. — Frank Rosenblatt and the perceptron. Image source: Computer History Museum.

What does an artificial neuron calculate?

Each input value (x) has an associated weight (w). The neuron multiplies the inputs by their weights, adds the results, and then uses an activation function to produce an output.

The weights determine how the inputs contribute to the output. During learning, these and other learnable numerical values, such as biases, are adjusted: we call them the model's parameters.

A perceptron's output of 0 or 1 represents one of two categories. Comparing it with the correct label tells us whether the prediction was right.

Inputs x are multiplied by weights w; the weighted sum passes through activation function f to produce output y. — Input → weighted sum → activation → output.

Try it!

How does a weight change the decision?

In this simple example, x₁ = 1 and x₂ = 1. If the weighted sum is at least 1, the output is 1; otherwise, it is 0. Move the sliders!

First weight (w₁): 0.4 Second weight (w₂): 0.4

Output:

This example illustrates the calculation. Adjusting the sliders manually is not automatic learning.

Multilayer neural networks and their parameters

The next step was to connect multiple artificial neurons and process information through several successive layers. Increasing computing capacity also made it possible to train more complex networks.

Hidden layers sit between the input and output layers. They process each other's outputs: later layers can build more complex patterns from the simpler features produced by earlier ones.

The word “deep” refers to the many successive processing layers. Behind the scenes, there are still mathematical operations and learnable parameters.

01

Input

The data to be processed, such as the pixel values of an image.

02

Hidden layers

Successive transformations of the data using learned weights.

03

Output

The result for the task, such as a predicted category.

From edges to recognizing a face

The early layers of an image recognition network can process simple visual features such as edges and contrasts. Subsequent layers can combine these into shapes and details, and later into more complex object or face patterns.

The network does not “see” the image through human eyes. It transforms numerical values and uses relationships learned during training.

Layers of face recognition: an input face image is transformed into edge patterns, facial features, and then patterns representing a complete face. — Successive levels of features in a face recognition example. Source: NVIDIA.

Learning from errors

Backpropagation made effective training of multilayer networks widely practical.

During training, we compare the network's answer with the desired result. Working backward from the output error, backpropagation calculates how the error depends on each parameter. An optimization algorithm uses this information to adjust the weights.

Repeating the training process many times aims to reduce the error. The finished model can then be tested on new data it did not encounter during training.

Further explanation: IBM – how backpropagation works.

The animation's opening frame: a neural network processing a cat's image, with a prediction of 0.7 at the output. — In the dog–cat example animation, the thickness of the connections represents the weights, and the output value illustrates the estimated probability of the cat category.

01

Forward pass

The input image passes through the network, producing a prediction.

02

Error calculation

The prediction is compared with the training example's correct label.

03

Weight adjustment

Working backward, we calculate how sensitive the error is to each parameter, then adjust the parameters.

What does the parameter count tell us?

Model size is often described by the number of parameters and layers. Layer width, meaning the number of neurons in each layer, also matters.

A higher parameter count means more learnable numerical values. On its own, however, it does not prove that a model will perform better on a particular task.

Performance is influenced by the network's architecture, the amount and quality of training data, the activation functions, the training procedure and the nature of the task.

The size of a dog–cat image recognizer also depends on the chosen architecture. There is no universal requirement for every such model to have at least one million parameters.

Different tasks, different networks

Many neural network architectures have emerged since the perceptron. Their structure depends on the problem we want to solve: processing sequences and time-based data, recognizing images, compressing data or creating new content.

A schematic comparison of network and machine learning architectures, including convolutional, generative and residual networks. — Different models and connection structures. The SVM shown in the illustration is a machine learning method, not a neural network. Image source credited in the document: deep-learning-site.com.

01

Image processing

Recognizing visual features, shapes and objects.

02

Sequences

Processing related language or time-based data.

03

Transformation

Compression, pattern recognition and generating new content.

Which “engine” should we use?

Many AI applications let us choose between several models. Within the same application, these are different “engines”, optimized for different capabilities, speeds and types of task.

Accuracy

Reasoning

A problem with several steps may call for a model that handles conditions and relationships consistently.

Response time

Speed

For a simple question or a short rewrite, a quick answer may matter more than complex analysis.

Specialization

Code and mathematics

Performance in programming, debugging and calculations can vary between models.

Context

Reading comprehension

With long documents, it matters how much text a model can handle and how well it follows the details.

Creation

Creativity

Style, variety and following instructions matter when brainstorming, writing and creating teaching materials.

Configuration

Tool use

The availability of web search, file analysis and other tools also depends on the application's and account's settings.

The resource requirements of deep learning

From an artificial neuron handling a few inputs, we have progressed to systems with many billions of parameters that process text, images and sound. The successes of deep learning involve significant data and computing requirements.

Training

Creating the model

Training large models can require high-performance graphics processors, long runtimes and substantial energy consumption. Collecting, cleaning and preparing the data is also a major cost.

Training duration depends on the model's size, the amount of data and the available hardware; the largest models may take months to train.

Use · inference

Running the finished model

Generating one answer usually requires far fewer resources than training the entire model. With high traffic, the total cost of use can still be substantial.

The cost depends on the chosen model, input and output lengths, tool use and the application's pricing.

Your choice of model matters

Simple task

“What is the capital of France?”

A quick model with lower resource requirements is often sufficient for a short question about a well-known fact.

Always using the largest model for simple tasks may needlessly increase waiting time and resource use.

Complex task

“Analyze this eighty-page application, identify the contradictions, and then suggest improvements based on the EU evaluation criteria”

Finding contradictions and suggesting improvements based on EU evaluation criteria requires several steps, handling a long text and careful reasoning.

The model's capabilities, clearly defined criteria and human review of the result all matter here.

A confident answer can still be wrong

Trained models produce estimates. Their performance is limited by their training data, methods and differences in new situations. Better algorithms can reduce errors, but they do not guarantee correct answers in every situation.

Object recognition applications often display a numerical value beside a category. This is the model's confidence score; it does not necessarily match the actual probability that the prediction is correct.

Text answers can also be wrong, even when no confidence value is shown. Fluent wording is not evidence of correctness.

Object recognition of people and a boat on a sailing vessel, with person and boat labels and various confidence scores. — The object recognizer assigns categories and scores to parts of the image. Image source: Arteries.

When and how should we check?

Asking ChatGPT

How can we trust AI's answers if it is often wrong?

Answer

You shouldn't trust it blindly, but it can still be a very useful tool. Think of it as an extremely knowledgeable, fast and helpful assistant who usually answers well but can sometimes be confidently wrong.

The level of trust also depends on the type of question involved.

01

High reliability

Mathematics (if there is no calculation error), programming, grammar, general scientific knowledge and explaining concepts.

02

Medium reliability

Historical details, lesser-known facts and professional summaries.

03

Lower reliability without external verification

Breaking news, current legislation, stock market data, rare statistics, specific quotations and exact references.

Knowledge check

Check what you have learned!

Select one answer for each question, then view your score and the explanations.

1How are AI, machine learning and deep learning related? The three terms mean exactly the same thing. Deep learning is part of machine learning, which is part of AI. Every AI system uses deep learning.

2What changes when a perceptron learns? The training example's correct label changes to match the model's prediction. The neuron becomes a real biological nerve cell. The learnable connection weights are adjusted based on errors.

3What does backpropagation do? It works backward from the output error to calculate how the error depends on the parameters. It deletes all incorrect training images. It guarantees that the finished model will never make another error.

4Which statement about parameter counts is correct? More parameters guarantee better results on every task. The count is an important size metric, but architecture and training data also affect performance. The parameter count is the same as the number of users.

5What does a high confidence score from an image recognizer mean? The model strongly supports that category, but the prediction may be wrong. The result is certainly correct and does not need checking. The model understands the image in the same way as a human.

6How should you choose a model to analyze a long application? Based only on the model's name. Always choose the model that gives the shortest answer. Based on text handling, reasoning, tools, cost and the results of your own trial.

Contributors and attachments

Credits and copyright

Author: Mária HartyányiiTStudy Hungary Training and Research Centre
Reviewer: Zsolt LengyeliTStudy Hungary Training and Research Centre
Project: 2025-1-LT01-KA220-VET-000362456
Course: M1-T1-MLC2

Attachments

Download the English course materialm1-t1-mlc2-text-en.pdf Download the practical taskm1-t1-mlc2-exc-en.pdf

Creative Commons Attribution-ShareAlike emblem

This work may be reused and adapted under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).

Funded by the European Union. Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the National Agency.