Mastering Embedded Linux Development: Craft fast and reliable embedded solutions with Linux 6.6 and The Yocto Project 5.0 (Scarthgap), 4th Edition / Linux: Linux 6.6 Yocto Project 5.0 (Scarthgap), 4-
: 2025
: Vasquez Frank, Simmonds Chris / ,
: Packt Publishing
ISBN: 978-1-80323-259-1
: Expert Insight
:
: PDF
: (eBook)
:
: 710
: Written by Frank Vasquez, an embedded Linux expert, this new edition enables you to harness the full potential of Linux to create versatile and robust embedded solutions
All formats include a free PDF and an invitation to the Embedded System Professionals community
Key Features
Learn how to develop and configure reliable embedded Linux devices
Discover the latest enhancements in Linux 6.6 and the Yocto Project 5.0, codename Scarthgap
Explore different ways to debug and profile your code in both user space and the Linux kernel
Book Description
Mastering Embedded Linux Development is designed to be both a learning resource and a reference for your embedded Linux projects.
In this fourth edition, you'll learn the fundamental elements that underpin all embedded Linux projects: the toolchain, the bootloader, the kernel, and the root filesystem. First, you will download and install a pre-built toolchain. After that, you will cross-compile each of the remaining three elements from scratch and learn to automate the process using Buildroot and the Yocto Project. The book progresses with coverage of over-the-air software updates and rapid prototyping with add-on boards. Two new chapters tackle modern development practices, including Python packaging and deploying containerized applications. These are followed by a chapter on writing multithreaded code and another on techniques to manage memory efficiently. The final chapters demonstrate how to debug your code, whether it resides in user space or in the Linux kernel itself. In addition to GNU debugger (GDB), the book also covers the different tracers and profilers that are available for Linux so that you can quickly pinpoint any performance bottlenecks in your system.
By the end of this book, you will be able to create efficient and secure embedded devices with Linux that will delight your users.
What you will learn
Cross-compile embedded Linux images with Buildroot and Yocto
Enable Wi-Fi and Bluetooth connectivity with a Yocto board support package
Update IoT devices securely in the field with Mender or balena
Prototype peripheral additions by connecting add-on boards, reading schematics, and coding test programs
Deploy containerized software applications on edge devices with Docker
Debug devices remotely using GDB and measure the performance of systems using tools like perf and ply
Who this book is for
If you are a systems software engineer or system administrator who wants to learn how to apply Linux to embedded devices, then this book is for you. The book is also for embedded software engineers accustomed to programming low-power microcontrollers and will help them make the leap to a high-speed system-on-chips that can run Linux. Anyone who develops hardware for Linux will find something useful in this book. But before you get started, you will need a solid grasp of the POSIX standard, C programming, and shell scripting.
, , Linux, Linux
, Linux
Linux 6.6 Yocto Project 5.0 Scarthgap
, Linux
Mastering Embedded Linux Development , embedded Linux.
, embedded Linux: , , . . Buildroot Yocto Project. . , Python . , , , . , , , Linux. GNU debugger (GDB), , Linux, .
, Linux, .
Linux Buildroot Yocto
Wi-Fi Bluetooth Yocto board
Mender balena
,
Docker
GDB , perf ply
- , Linux , . - , , , Linux. , Linux, - . , POSIX, .
Preface xxix
Part I: Elements of Embedded Linux 1
Chapter 1: Starting Out 3
Choosing Linux ................................................................................................................... 4
When not to choose Linux .................................................................................................. 5
Meeting the players ............................................................................................................ 6
Moving through the project life cycle .................................................................................. 7
The four elements of embedded Linux ................................................................................ 8
Navigating open source ....................................................................................................... 9
Licenses 9
Selecting hardware for embedded Linux ............................................................................ 11
Obtaining the hardware for this book ............................................................................... 12
The Raspberry Pi 4 12
The BeaglePlay 13
QEMU 14
Provisioning your development environment .................................................................... 15
Summary .......................................................................................................................... 16
Chapter 2: Learning about Toolchains 17
Technical requirements .................................................................................................... 18
Introducing toolchains ..................................................................................................... 18
Types of toolchains 20
CPU architectures 21
Choosing the C library 23
Finding a toolchain ........................................................................................................... 24
Anatomy of a toolchain ..................................................................................................... 26
Finding out about your cross compiler 27
sysroot, library, and header files 29
Other tools in the toolchain 29
Looking at the components of the C library 30
Linking with libraries static and dynamic linking ........................................................... 31
Static libraries 32
Shared libraries 34
Understanding shared library version numbers 35
Art of cross-compiling ....................................................................................................... 36
Simple makefiles 37
Autotools 37
An example: SQLite 39
Package configuration 41
Problems with cross-compiling 42
CMake 43
Summary .......................................................................................................................... 46
Further study .................................................................................................................... 46
Chapter 3: All about Bootloaders 47
Technical requirements .................................................................................................... 47
What does a bootloader do? .............................................................................................. 48
Boot sequence ................................................................................................................... 49
Phase 1 ROM code 50
Phase 2 Secondary Program Loader 51
Phase 3 Tertiary Program Loader 52
Moving from the bootloader to the kernel ........................................................................ 53
Introducing device trees ................................................................................................... 54
Device tree basics 54
reg property 56
Labels and interrupts 56
Device tree include files 58
Compiling a device tree 60
U-Boot .............................................................................................................................. 60
Building U-Boot 61
Obtaining a 32-bit toolchain 61
Building U-Boot SPL for R5 62
Generating an image for the R5 68
Building TF-A for the main A53 CPU 69
Building U-Boot for the main A53 CPU 70
Installing U-Boot 72
Using U-Boot 76
Environment variables 77
Boot image format 77
Loading images 79
Booting Linux 81
Automating the boot with U-Boot scripts 81
Porting U-Boot to a new board 81
Board-specific files 83
Configuring header files 85
Building and testing 86
Falcon mode 89
Summary .......................................................................................................................... 89
Chapter 4: Configuring and Building the Kernel 91
Technical requirements .................................................................................................... 92
What does the kernel do? .................................................................................................. 92
Choosing a kernel ............................................................................................................. 94
Kernel development cycle 94
Stable and long-term support releases 95
Vendor support 96
Licensing 97
Best practices 97
Configuring the kernel ...................................................................................................... 98
Getting the source 98
Understanding kernel configuration Kconfig 99
Using LOCALVERSION to identify your kernel 104
When to use kernel modules 105
Compiling with Kbuild ................................................................................................... 106
Finding out which kernel target to build 106
Build artifacts 107
Compiling device trees 109
Compiling modules 109
Cleaning kernel sources 110
Building and booting the kernel ...................................................................................... 110
Building a kernel for the Raspberry Pi 4 110
Booting the Raspberry Pi 4 112
Building a kernel for the BeaglePlay 113
Booting the BeaglePlay 114
Building a kernel for QEMU 116
Booting QEMU 116
Observing the kernel boot process ................................................................................... 117
Kernel panic 117
Early user space 117
Kernel messages 118
Kernel command line 119
Porting Linux to a new board .......................................................................................... 120
A new device tree 120
Summary ......................................................................................................................... 122
Further study ................................................................................................................... 123
Chapter 5: Building a Root Filesystem 125
Technical requirements ................................................................................................... 126
What should be in the root filesystem? ............................................................................ 126
Directory layout 127
Staging directory 129
POSIX file access permissions 130
File ownership permissions in the staging directory 132
Programs for the root filesystem 132
init program 133
Shell 133
Utilities 133
BusyBox to the rescue! 134
Building BusyBox 135
ToyBox an alternative to BusyBox 136
Libraries for the root filesystem 136
Reducing size by stripping 138
Device nodes 139
Proc and sysfs filesystems 140
Mounting filesystems 141
Kernel modules 142
Transferring the root filesystem to the target .................................................................. 142
Creating a boot initramfs ................................................................................................. 143
Standalone initramfs 143
Booting an initramfs 144
Booting with QEMU 144
Booting the BeaglePlay 144
Mounting proc 145
Building an initramfs into the kernel image 145
Building an initramfs using a device table 146
Old initrd format 148
init program .................................................................................................................... 148
Starting a daemon process 149
Configuring user accounts ............................................................................................... 149
Adding user accounts to the root filesystem 151
A better way of managing device nodes ........................................................................... 152
Using devtmpfs 152
Using mdev 153
Are static device nodes so bad after all? 154
Configuring the network ................................................................................................. 154
Network components for glibc 155
Creating filesystem images with device tables ................................................................. 156
Booting the BeaglePlay 157
Mounting the root filesystem using NFS ......................................................................... 158
Testing with the BeaglePlay 159
Problems with file permissions 160
Using TFTP to load the kernel ......................................................................................... 160
Summary ......................................................................................................................... 162
Further study ................................................................................................................... 162
Part II: Building Embedded Linux Images 165
Chapter 6: Selecting a Build System 167
Technical requirements ................................................................................................... 167
Comparing build systems ............................................................................................... 168
Distributing binaries ...................................................................................................... 170
Introducing Buildroot ..................................................................................................... 170
Background 170
Stable releases and long-term support 171
Installing 171
Configuring 171
Running 172
Targeting real hardware 174
Creating a custom BSP 175
U-Boot 175
Linux 176
Build 177
Adding your own code 180
Overlays 180
Adding a package 181
License compliance 183
Introducing The Yocto Project ......................................................................................... 183
Background 183
Stable releases and supports 185
Installing The Yocto Project 185
Configuring 186
Building 187
Running the QEMU target 188
Layers 188
BitBake and recipes 191
Customizing images via local.conf 194
Writing an image recipe 195
Creating an SDK 196
The license audit 198
Summary ........................................................................................................................ 198
Further study .................................................................................................................. 199
Chapter 7: Developing with Yocto 201
Technical requirements .................................................................................................. 202
Building on top of an existing BSP .................................................................................. 202
Building an existing BSP 203
Controlling Wi-Fi 210
Controlling Bluetooth 213
Adding a custom layer 216
Capturing changes with devtool ...................................................................................... 219
Development workflows 220
Creating a new recipe 221
Modifying the source built by a recipe 223
Upgrading a recipe to a newer version 225
Building your own distro ................................................................................................ 229
When and when not to 229
Creating a new distro layer 229
Configuring your distro 230
Adding more recipes to your distro 230
Runtime package management 231
Provisioning a remote package server ............................................................................. 232
Summary ........................................................................................................................ 235
Further study .................................................................................................................. 235
Chapter 8: Yocto under the Hood 237
Technical requirements .................................................................................................. 237
Decomposing Yoctos architecture and workflow ........................................................... 238
Metadata 240
Build tasks 241
Image generation 242
Separating metadata into layers ..................................................................................... 244
Troubleshooting build failures ....................................................................................... 246
Isolating errors 247
Dumping the environment 247
Reading the task log 248
Adding more logging 249
Running commands from devshell 250
Graphing dependencies 250
Understanding BitBake syntax and semantics ................................................................ 252
Tasks 252
Dependencies 253
Inter-task dependencies 253
Build-time dependencies 253
Runtime dependencies 254
Variables 254
Assignment and expansion 254
Appending and prepending 255
Overrides 256
Inline Python 257
Functions 258
Shell 258
Python 258
RDEPENDS revisited 260
Summary ......................................................................................................................... 261
Further study .................................................................................................................. 262
Part III: System Architecture and Design Decisions 263
Chapter 9: Creating a Storage Strategy 265
Technical requirements .................................................................................................. 266
Storage options ............................................................................................................... 266
NOR flash 267
NAND flash 268
Managed flash 270
The MultiMediaCard and Secure Digital cards 270
eMMC 271
Other types of managed flash 271
Accessing flash memory from the bootloader ................................................................. 272
U-Boot and NOR flash 272
U-Boot and NAND flash 272
U-Boot and MMC, SD, and eMMC 273
Accessing flash memory from Linux ................................................................................ 273
Memory technology devices 274
MTD partitions 274
MTD device drivers 278
The MTD character device, mtd 278
The MTD block device, mtdblock 279
Logging kernel oops to MTD 280
Simulating NAND memory 280
The MMC block driver 280
Filesystems for flash memory ......................................................................................... 280
Flash translation layers 281
Filesystems for NOR and NAND flash memory ................................................................ 282
JFFS2 282
Summary nodes 283
Clean markers 284
Creating a JFFS2 filesystem 284
YAFFS2 285
Creating a YAFFS2 filesystem 286
UBI and UBIFS 286
UBI 287
UBIFS 289
Filesystems for managed flash ......................................................................................... 291
Flashbench 291
Discard and TRIM 292
Ext4 294
F2FS 295
FAT16/32 295
Read-only compressed filesystems .................................................................................. 296
SquashFS 296
Temporary filesystems .................................................................................................... 297
Making the root filesystem read-only ............................................................................. 298
Filesystem choices .......................................................................................................... 299
Summary ....................................................................................................................... 300
Further study ................................................................................................................. 300
Chapter 10: Updating Software in the Field 301
Technical requirements .................................................................................................. 302
From where do updates originate? .................................................................................. 302
What to update ............................................................................................................... 302
Bootloader 303
Kernel 303
Root filesystem 304
System applications 304
Device-specific data 304
Components that need to be updated 304
Basics of software updates .............................................................................................. 305
Making updates robust 305
Making updates fail-safe 306
Making updates secure 308
Types of update mechanism ........................................................................................... 309
Symmetric image update 310
Asymmetric image update 311
Atomic file updates 312
OTA updates ..................................................................................................................... 314
Using Mender for local updates ....................................................................................... 315
Building the Mender client 315
Installing an update with Mender 318
Using Mender for OTA updates ........................................................................................ 321
Using SWUpdate for local updates .................................................................................. 328
Summary ........................................................................................................................ 329
Chapter 11: Interfacing with Device Drivers 331
Technical requirements .................................................................................................. 332
Role of device drivers ...................................................................................................... 332
Character devices ............................................................................................................ 333
Block devices .................................................................................................................. 336
Network devices ............................................................................................................. 337
Finding out about drivers at runtime .............................................................................. 339
Getting information from sysfs 340
Devices /sys/devices 340
Drivers /sys/class 341
Block drivers /sys/block 342
Finding the right device driver ........................................................................................ 343
Device drivers in user space ............................................................................................ 344
GPIO 345
Handling interrupts from GPIO 347
LEDs 350
I2C 351
SPI 353
Writing a kernel device driver ......................................................................................... 354
Designing a character driver interface 354
Anatomy of a device driver 356
Compiling kernel modules 359
Loading kernel modules 360
Discovering the hardware configuration ......................................................................... 361
Device trees 362
Platform data 362
Linking hardware with device drivers 363
Summary ........................................................................................................................ 366
Further study .................................................................................................................. 366
Chapter 12: Prototyping with Add-On Boards 369
Technical requirements .................................................................................................. 370
Mapping schematics to pins ........................................................................................... 370
Reading schematics 371
Installing Debian on the BeaglePlay 373
Prototyping with add-on boards ..................................................................................... 375
Testing hardware peripherals .......................................................................................... 377
Attaching the GNSS Click 7 add-on board 377
Receiving NMEA messages 379
Attaching the Environment Click add-on board 381
Reading sensor values 382
Attaching the OLED C Click add-on board 383
Displaying an animation 384
Summary ........................................................................................................................ 385
Further study .................................................................................................................. 385
Chapter 13: Starting Up The init Program 387
Technical requirements .................................................................................................. 387
After the kernel has booted ............................................................................................. 388
Introducing the init programs ........................................................................................ 389
BusyBox init .................................................................................................................... 390
Buildroot init scripts 391
System V init .................................................................................................................... 391
inittab 393
The init.d scripts 396
Adding a new daemon 396
Starting and stopping services 398
systemd .......................................................................................................................... 399
Building systemd with The Yocto Project and Buildroot 399
Introducing targets, services, and units 399
Units 400
Services 401
Targets 402
How systemd boots the system 402
Adding your own service 403
Adding a watchdog 405
Implications for embedded Linux 406
Summary ........................................................................................................................ 406
Further study .................................................................................................................. 406
Chapter 14: Managing Power 407
Technical requirements ................................................................................................. 408
Measuring power usage ................................................................................................. 408
Scaling the clock frequency .............................................................................................. 411
CPUFreq driver 412
Using CPUFreq 414
Selecting the best idle state .............................................................................................. 417
CPUIdle driver 418
Tickless operation 419
Powering down peripherals ............................................................................................ 419
Putting the system to sleep .............................................................................................. 421
Power states 421
Wakeup events 422
Timed wakeups from the real-time clock 423
Summary ........................................................................................................................ 424
Further study .................................................................................................................. 425
Part IV: Developing Applications 427
Chapter 15: Packaging Python 429
Technical requirements .................................................................................................. 430
Retracing the origins of Python packaging ..................................................................... 430
distutils 431
setuptools 431
setup.py 432
Installing Python packages with pip ............................................................................... 434
requirements.txt 437
Managing Python virtual environments with venv ......................................................... 440
Installing precompiled binaries with conda ................................................................... 443
Environment management 444
Package management 446
Summary ........................................................................................................................ 449
Further study .................................................................................................................. 449
Chapter 16: Deploying Container Images 451
Technical requirements ................................................................................................... 451
Getting Docker 452
What is DevOps? ............................................................................................................. 453
Continuous integration and continuous deployment 453
Infrastructure as code 454
Security is a shared responsibility 454
Monitoring and observability 455
Continuous improvement 455
Transparency 455
DevOps and Embedded Linux ......................................................................................... 455
Continuous integration and cross-compilation 456
Automated testing on real hardware 456
Continuous delivery and OTA updates 457
Infrastructure as code and build systems 457
Securing edge devices 458
Monitoring and observability of edge devices 458
Deploying Python applications with Docker .................................................................. 458
Anatomy of a Dockerfile 459
Building a Docker image 462
Running a Docker image 463
Fetching a Docker image 463
Publishing a Docker image 464
Cleaning up 465
Setting up a CI/CD pipeline for a Python application ...................................................... 467
Creating a Dockerfile 467
Creating a GitHub Actions workflow 469
Pulling and running the latest image 472
Adding Docker to a Yocto image ...................................................................................... 478
Adding the meta-virtualization layer 478
Installing Docker 479
Verifying the Docker daemon is running 481
Updating software with Docker ...................................................................................... 482
Creating an account 483
Creating an application 484
Adding a device 484
Installing the CLI 487
Pushing a project 489
Summary ........................................................................................................................ 492
Further study .................................................................................................................. 492
Chapter 17: Learning about Processes and Threads 493
Technical requirements .................................................................................................. 494
Process or thread? ........................................................................................................... 494
Processes ......................................................................................................................... 496
Creating a new process 496
Terminating a process 497
Running a different program 499
Daemons 502
Inter-process communication 502
Message-based IPC 503
Unix (or local) sockets 503
FIFOs and named pipes 504
POSIX message queues 504
Summary of message-based IPC 504
Shared memory-based IPC 505
POSIX shared memory 505
Threads ........................................................................................................................... 508
Creating a new thread 508
Terminating a thread 510
Compiling a program with threads 511
Inter-thread communication 511
Mutual exclusion 511
Changing conditions 512
Partitioning the problem 514
ZeroMQ ............................................................................................................................ 515
Getting pyzmq 516
Messaging between processes 516
Messaging within processes 518
Scheduling ...................................................................................................................... 520
Fairness versus determinism 521
Time-shared policies 521
Niceness 522
Real-time policies 523
Choosing a policy 524
Choosing a real-time priority 524
Summary ........................................................................................................................ 525
Further study .................................................................................................................. 525
Chapter 18: Managing Memory 527
Technical requirements .................................................................................................. 528
Virtual memory basics .................................................................................................... 528
Kernel-space memory layout .......................................................................................... 529
How much memory does the kernel use? 531
User-space memory layout ............................................................................................. 533
Process memory map ...................................................................................................... 535
Swapping ......................................................................................................................... 537
Swapping to compressed memory (zram) 537
Mapping memory with mmap ........................................................................................ 538
Using mmap to allocate private memory 538
Using mmap to share memory 539
Using mmap to access device memory 539
How much memory does my application use? ................................................................ 540
Per-process memory usage .............................................................................................. 541
Using top and ps 541
Using smem 542
Other tools to consider 544
Identifying memory leaks ............................................................................................... 545
mtrace 545
Valgrind 546
Running out of memory .................................................................................................. 548
Summary ........................................................................................................................ 550
Further study ................................................................................................................... 551
Part V: Debugging and Optimizing Performance 553
Chapter 19: Debugging with GDB 555
Technical requirements .................................................................................................. 555
GNU debugger ................................................................................................................. 556
Preparing to debug .......................................................................................................... 557
Debugging applications .................................................................................................. 558
Remote debugging using gdbserver 558
Setting up The Yocto Project for remote debugging 559
Setting up Buildroot for remote debugging 560
Starting to debug 561
Connecting GDB and gdbserver 561
Setting the sysroot 562
GDB command files 564
Overview of GDB commands 564
Running to a breakpoint 566
Extending GDB with Python 567
Native debugging 571
The Yocto Project 571
Buildroot 572
Just-in-time debugging ................................................................................................... 572
Debugging forks and threads .......................................................................................... 573
Core files ......................................................................................................................... 573
Using GDB to look at core files 575
GDB user interfaces ......................................................................................................... 576
Terminal User Interface 576
Data Display Debugger 577
Visual Studio Code 578
Debugging kernel code ................................................................................................... 579
Debugging kernel code with kgdb 579
A sample debug session 580
Debugging early code 584
Debugging modules 584
Debugging kernel code with kdb 585
Looking at an Oops 587
Preserving the Oops 590
Summary ......................................................................................................................... 591
Further study ................................................................................................................... 591
Chapter 20: Profiling and Tracing 593
Technical requirements .................................................................................................. 594
Observer effect ................................................................................................................ 594
Beginning to profile ........................................................................................................ 596
Profiling with top 596
Profiling with GDB 598
Introducing perf ............................................................................................................. 598
Configuring the kernel for perf 599
Building perf with The Yocto Project 600
Building perf with Buildroot 600
Profiling with perf 601
Call graphs 603
perf annotate 604
Tracing events ................................................................................................................. 606
Introducing Ftrace .......................................................................................................... 606
Preparing to use Ftrace 607
Using Ftrace 607
Dynamic Ftrace and trace filters 609
Trace events 611
Using LTTng .................................................................................................................... 612
LTTng and The Yocto Project 613
LTTng and Buildroot 613
Using LTTng for kernel tracing 613
Using eBPF ...................................................................................................................... 616
Configuring the kernel for eBPF 616
Building ply with Buildroot 618
Using ply 618
Using Valgrind ................................................................................................................. 621
Callgrind 621
Helgrind 622
Using strace .................................................................................................................... 622
Summary ........................................................................................................................ 625
Further study .................................................................................................................. 626
Chapter 21: Real-Time Programming 627
Technical requirements .................................................................................................. 628
What is real time? ........................................................................................................... 628
Identifying sources of non-determinism ......................................................................... 630
Understanding scheduling latency ................................................................................. 632
Kernel preemption .......................................................................................................... 632
Real-time Linux kernel (PREEMPT_RT) 633
Threaded interrupt handlers 634
Preemptible kernel locks ................................................................................................. 636
Getting the PREEMPT_RT patches 637
The Yocto Project and PREEMPT_RT 638
High-resolution timers ................................................................................................... 639
Avoiding page faults ........................................................................................................ 640
Interrupt shielding ......................................................................................................... 641
Measuring scheduling latencies ...................................................................................... 642
cyclictest 642
Using Ftrace 643
Combining cyclictest and Ftrace 645
xxviii Table of Contents
Summary ........................................................................................................................ 646
Further study .................................................................................................................. 647
Other Books You May Enjoy 651
Index 655
