(2025/2026) Artificial Neural Networks and Deep Learning

The notes are taken from the books required for the course:

You can view/download the PDF here. In the notes folder, you can also see the source code.

For any issue, use the appropriate section.

Correlated Projects

During the Artificial Neural Networks and Deep Learning for HPC course, me and my other three colleagues Alberto Ondei, Abdullah Javed, Filippo Barbari created two projects:

  1. Time Series Classification. Deep Neural Netowrk (TCN + BiLSTM with Attention) model on multivariate time-series classification.
  2. Image Classification. Histopathology image classification to predict molecular subtypes. Includes datasets, EDA and preprocessing notebooks, patch extraction, baseline models, and training utilities for quick experiments and reproducible evaluation.

Course Syllabus

According to the official course syllabus:

  1. From the Perceptron to Neural Networks and the Feedforward architecture
  2. Backpropagation and Neural Networks training algorithms, e.g., Adagrad, adam, etc.
  3. Best practices in neural network training: overfitting and cross-validation, stopping criteria, weight decay, dropout, data resampling and augmentation.
  4. Image Classification problem and Neural Networks
  5. Recurrent Neural Networks and other relevant architectures such as (Sparse) Neural Autoencoders
  6. Theoretical results: Neural Networks as universal approximation tools, vanishing and exploding gradients, etc.
  7. Introduction to the Deep Learning paradigm and its main differences with respect to classical Machine Learning
  8. Convolutional Neural Networks (CNN) architecture
  9. The breakthrough of CNN and their interpretation
  10. CNN training and data-augmentation
  11. Structural learning, Long-Short Term Memories, and their applications to text and speech
  12. Autoencoders and data embedding, word2vec, variational autoencoders
  13. Transfer Learning for pre-trained Deep models
  14. Extended models including Fully Convolutional CNN, networks for image segmentation (U-net) and object detection (e.g., R-CNN, YOLO )
  15. Generative Models (e.g., Generative Adversarial Networks)