본문 바로가기 주메뉴 바로가기
22대 대한민국 국회 국회정보길라잡이 국회의원검색
회원가입 로그인
HOME

정보검색

소장정보 검색

국회도서관 홈으로 정보검색 소장정보 검색
결과 내 검색 동의어 포함
결과 내 검색 동의어 포함

-

목차보기


Foreword
Preface
Authors
Contributors
Chapter 1 ◾ Deep Learning and Transformers: An Introduction
1.1 Deep Learning: A Historic Perspective
1.2 Transformers and Taxonomy
1.2.1 Modified Transformer Architecture
1.2.1.1 Transformer block changes
1.2.1.2 Transformer sublayer changes
1.2.2 Pre-training Methods and Applications
1.3 Resources
1.3.1 Libraries and Implementations
1.3.2 Books
1.3.3 Courses, Tutorials, and Lectures
1.3.4 Case Studies and Details
Chapter 2 ◾ Transformers: Basics and Introduction
2.1 Encoder-Decoder Architecture
2.2 Sequence-to-Sequence
2.2.1 Encoder
2.2.2 Decoder
2.2.3 Training
2.2.4 Issues with RNN-Based Encoder-Decoder
2.3 Attention Mechanism
2.3.1 Background
2.3.2 Types of Score-Based Attention
2.3.2.1 Dot product (multiplicative)
2.3.2.2 Scaled dot product or multiplicative
2.3.2.3 Linear, MLP, or Additive
2.3.3 Attention-Based Sequence-to-Sequence
2.4 Transformer
2.4.1 Source and Target Representation
2.4.1.1 Word embedding
2.4.1.2 Positional encoding
2.4.2 Attention Layers
2.4.2.1 Self-attention
2.4.2.2 Multi-head attention
2.4.2.3 Masked multi-head attention
2.4.2.4 Encoder-decoder multi-head attention
2.4.3 Residuals and Layer Normalization
2.4.4 Positionwise Feed-forward Networks
2.4.5 Encoder
2.4.6 Decoder
2.5 Case Study: Machine Translation
2.5.1 Goal
2.5.2 Data, Tools, and Libraries
2.5.3 Experiments, Results, and Analysis
2.5.3.1 Exploratory data analysis
2.5.3.2 Attention
2.5.3.3 Transformer
2.5.3.4 Results and analysis
2.5.3.5 Explainability
Chapter 3 ◾ Bidirectional Encoder Representations from Transformers (BERT)
3.1 BERT
3.1.1 Architecture
3.1.2 Pre-Training
3.1.3 Fine-Tuning
3.2 BERT Variants
3.2.1 RoBERTa
3.3 Applications
3.3.1 TaBERT
3.3.2 BERTopic
3.4 BERT Insights
3.4.1 BERT Sentence Representation
3.4.2 BERTology
3.5 Case Study: Topic Modeling with Transformers
3.5.1 Goal
3.5.2 Data, Tools, and Libraries
3.5.2.1 Data
3.5.2.2 Compute embeddings
3.5.3 Experiments, Results, and Analysis
3.5.3.1 Building topics
3.5.3.2 Topic size distribution
3.5.3.3 Visualization of topics
3.5.3.4 Content of topics
3.6 Case Study: Fine-tuning BERT
3.6.1 Goal
3.6.2 Data, Tools, and Libraries
3.6.3 Experiments, Results, and Analysis
Chapter 4 ◾ Multilingual Transformer Architectures
4.1 Multilingual Transformer Architectures
4.1.1 Basic Multilingual Transformer
4.1.2 Single-Encoder Multilingual NLU
4.1.2.1 mBERT
4.1.2.2 XLM
4.1.2.3 XLM-RoBERTa
4.1.2.4 ALM
4.1.2.5 Unicoder
4.1.2.6 INFOXLM
4.1.2.7 AMBER
4.1.2.8 ERNIE-M
4.1.2.9 HITCL
4.1.3 Dual-Encoder Multilingual NLU
4.1.3.1 LaBSE
4.1.3.2 mUSE
4.1.4 Multilingual NLG
4.2 Multilingual Data
4.2.1 Pre-Training Data
4.2.2 Multilingual Benchmarks
4.2.2.1 Classification
4.2.2.2 Structure prediction
4.2.2.3 Question answering
4.2.2.4 Semantic retrieval
4.3 Multilingual Transfer Learning Insights
4.3.1 Zero-Shot Cross-Lingual Learning
4.3.1.1 Data factors
4.3.1.2 Model architecture factors
4.3.1.3 Model tasks factors
4.3.2 Language-Agnostic Cross-Lingual Representations
4.4 Case Study
4.4.1 Goal
4.4.2 Data, Tools, and Libraries
4.4.3 Experiments, Results, and Analysis
4.4.3.1 Data preprocessing
4.4.3.2 Experiments
Chapter 5 ◾ Transformer Modifications
5.1 Transformer Block Modifications
5.1.1 Lightweight Transformers
5.1.1.1 Funnel-transformer
5.1.1.2 DeLighT
5.1.2 Connections between Transformer Blocks
5.1.2.1 RealFormer
5.1.3 Adaptive Computation Time
5.1.3.1 Universal transformers (UT)
5.1.4 Recurrence Relations between Transformer Blocks
5.1.4.1 Transformer-XL
5.1.5 Hierarchical Transformers
5.2 Transformers with Modified Multi-head Self-Attention
5.2.1 Structure of Multi-Head Self-Attention
5.2.1.1 Multi-head self-attention
5.2.1.2 Space and time complexity
5.2.2 Reducing Complexity of Self-Attention
5.2.2.1 Longformer
5.2.2.2 Reformer
5.2.2.3 Performer
5.2.2.4 Big Bird
5.2.3 Improving Multi-Head-Attention
5.2.3.1 Talking-heads attention
5.2.4 Biasing Attention with Priors
5.2.5 Prototype Queries
5.2.5.1 Clustered attention
5.2.6 Compressed Key-Value Memory
5.2.6.1 Luna: Linear Unified Nested Attention
5.2.7 Low-Rank Approximations
5.2.7.1 Linformer
5.3 Modifications for training task efficiency
5.3.1 ELECTRA
5.3.1.1 Replaced token detection
5.3.2 T5
5.4 Transformer submodule changes
5.4.1 Switch Transformer
5.5 Case Study: Sentiment Analysis
5.5.1 Goal
5.5.2 Data, Tools, and Libraries
5.5.3 Experiments, Results, and Analysis
5.5.3.1 Visualizing attention head weights
5.5.3.2 Analysis
Chapter 6 ◾ Pre-trained and Application-Specific Transformers
6.1 Text Processing
6.1.1 Domain-Specific Transformers
6.1.1.1 BioBERT
6.1.1.2 SciBERT
6.1.1.3 FinBERT
6.1.2 Text-to-Text Transformers
6.1.2.1 ByT5
6.1.3 Text Generation
6.1.3.1 GPT: Generative pre-training
6.1.3.2 GPT-2
6.1.3.3 GPT-3
6.2 Computer Vision
6.2.1 Vision Transformer
6.3 Automatic speech recognition
6.3.1 Wav2vec 2.0
6.3.2 Speech2Text2
6.3.3 HuBERT: Hidden Units BERT
6.4 Multimodal and Multitasking Transformer
6.4.1 Vision-and-Language BERT (VilBERT)
6.4.2 Unified Transformer (UniT)
6.5 Video Processing with TimeSformer
6.5.1 Patch Embeddings
6.5.2 Self-Attention
6.5.2.1 Spatiotemporal self-attention
6.5.2.2 Spatiotemporal attention blocks
6.6 Graph Transformers
6.6.1 Positional Encodings in a Graph
6.6.1.1 Laplacian positional encodings
6.6.2 Graph Transformer Input
6.6.2.1 Graphs without edge attributes
6.6.2.2 Graphs with edge attributes
6.7 Reinforcement Learning
6.7.1 Decision Transformer
6.8 Case Study: Automatic Speech Recognition
6.8.1 Goal
6.8.2 Data, Tools, and Libraries
6.8.3 Experiments, Results, and Analysis
6.8.3.1 Preprocessing speech data
6.8.3.2 Evaluation
Chapter 7 ◾ Interpretability and Explainability Techniques for Transformers
7.1 Traits of Explainable Systems
7.2 Related Areas that Impact Explainability
7.3 Explainable Methods Taxonomy
7.3.1 Visualization Methods
7.3.1.1 Backpropagation-based
7.3.1.2 Perturbation-based
7.3.2 Model Distillation
7.3.2.1 Local approximation
7.3.2.2 Model translation
7.3.3 Intrinsic Methods
7.3.3.1 Probing mechanism
7.3.3.2 Joint training
7.4 Attention and Explanation
7.4.1 Attention is Not an Explanation
7.4.1.1 Attention weights and feature importance
7.4.1.2 Counterfactual experiments
7.4.2 Attention is Not Not an Explanation
7.4.2.1 Is attention necessary for all tasks?
7.4.2.2 Searching for adversarial models
7.4.2.3 Attention probing
7.5 Quantifying Attention Flow
7.5.1 Information Flow as DAG
7.5.2 Attention Rollout
7.5.3 Attention Flow
7.6 Case Study: Text Classification with Explainability
7.6.1 Goal
7.6.2 Data, Tools, and Libraries
7.6.3 Experiments, Results, and Analysis
7.6.3.1 Exploratory data analysis
7.6.3.2 Experiments
7.6.3.3 Error analysis and explainability
Bibliography
Index

이용현황보기

Transformers for machine learning : a deep dive 이용현황 표 - 등록번호, 청구기호, 권별정보, 자료실, 이용여부로 구성 되어있습니다.
등록번호 청구기호 권별정보 자료실 이용여부
0003010378 006.32 -A23-3 서울관 서고(열람신청 후 1층 대출대) 이용가능

출판사 책소개

알라딘제공

Transformers are becoming a core part of many neural network architectures, employed in a wide range of applications such as NLP, Speech Recognition, Time Series, and Computer Vision. Transformers have gone through many adaptations and alterations, resulting in newer techniques and methods. Transformers for Machine Learning: A Deep Dive is the first comprehensive book on transformers.

Key Features:

  • A comprehensive reference book for detailed explanations for every algorithm and techniques related to the transformers.
  • 60+ transformer architectures covered in a comprehensive manner.
  • A book for understanding how to apply the transformer techniques in speech, text, time series, and computer vision.
  • Practical tips and tricks for each architecture and how to use it in the real world.
  • Hands-on case studies and code snippets for theory and practical real-world analysis using the tools and libraries, all ready to run in Google Colab.

The theoretical explanations of the state-of-the-art transformer architectures will appeal to postgraduate students and researchers (academic and industry) as it will provide a single entry point with deep discussions of a quickly moving field. The practical hands-on case studies and code will appeal to undergraduate students, practitioners, and professionals as it allows for quick experimentation and lowers the barrier to entry into the field.



Transformers are becoming a core part of many neural network architectures, employed in a wide range of applications such as NLP, Speech Recognition, Time Series, and Computer Vision. Transformers have gone through many adaptations and alterations, resulting in newer techniques and methods. This is the first comprehensive book on transformers.

추천서가 (다양한 추천 자료를 만나보세요)

권호기사보기

권호기사 목록 테이블로 기사명, 저자명, 페이지, 원문, 기사목차 순으로 되어있습니다.
기사명 저자명 페이지 원문 기사목차