Introduction to x64 ASM

1. Introduction to Assembly and x64 Architecture

What is Assembly Language?

Every program you’ve ever run, whether it’s a shell script, a Python interpreter, or a AAA game engine, eventually gets reduced to a stream of raw binary instructions that a CPU can execute directly. Assembly language is the thin, human-readable layer sitting right on top of that binary stream. Each assembly instruction maps almost one-to-one to a single machine instruction, so when you write mov rax, rbx, you’re really just writing a mnemonic for a specific sequence of bits that tells the processor “copy the contents of one register into another.” There’s no compiler doing clever things behind your back, no garbage collector, no runtime. What you write is (almost) exactly what runs.

That directness is also why assembly isn’t portable the way C or Python are. A high-level language gets compiled down to whatever instruction set the target CPU understands, so the same source file can, in principle, run on x86, ARM, or RISC-V. Assembly skips that translation step entirely, which means it’s tied to the specific architecture it was written for. Code written for x86 simply won’t run on an ARM chip, because the two architectures don’t just use different instruction encodings, they have different registers, different calling conventions, and different rules about how memory gets addressed. Learning assembly for one architecture doesn’t directly transfer to another, though the concepts (registers, stacks, flags, addressing modes) rhyme across all of them once you’ve internalized one.

This tutorial focuses on x64 assembly, sometimes called x86-64 or AMD64, which is the 64-bit extension of the x86 architecture. It’s what runs on essentially every modern Intel and AMD desktop, laptop, and server CPU. If you’ve ever wondered what’s actually happening when your compiled C program executes, or you’re trying to get a handle on how debuggers, exploit development, or performance-critical code actually works, this is the layer where those questions get answered.

2. x64 Registers

Before you can write a single useful instruction, you need to understand registers, because they’re where almost all the actual work happens. A register is a tiny, blazingly fast storage location built directly into the CPU itself, as opposed to RAM, which is comparatively enormous but much slower to reach. Every arithmetic operation, every comparison, every function call ultimately funnels data through registers, because the CPU’s execution units can only operate on values that are already sitting in them (or, for some instructions, values it can fetch from memory in the same step).

x64 gives you 16 general-purpose registers, each a full 64 bits wide. Some of them, like RAX and RCX, are holdovers from the 32-bit and 16-bit x86 era and carry historical names that hint at their traditional roles (accumulator, counter, and so on), even though in practice you can use most of them fairly interchangeably for general storage. The other eight, R8 through R15, were added when the architecture was extended to 64 bits and don’t carry any legacy baggage; they’re just extra general-purpose slots.

What makes this table worth studying closely, rather than skimming, is the aliasing. Each 64-bit register overlaps with smaller sub-registers that let you operate on just the low 32, 16, or 8 bits of the same physical storage. Write to EAX and you’re touching the lower half of RAX; write to AL and you’re touching just its lowest byte. This isn’t a coincidence or a naming convention, it’s how the hardware is literally wired: the smaller registers are views into the same physical bits as their 64-bit parent. One quirk worth remembering is that writing to a 32-bit register (like EAX) automatically zeroes out the upper 32 bits of the corresponding 64-bit register (RAX), while writing to a 16-bit or 8-bit alias leaves the untouched upper bits exactly as they were. That asymmetry has bitten more than one assembly programmer chasing a bug where a register seemed to contain “leftover garbage.”

Two of these registers deserve special attention because the CPU treats them as more than just “general-purpose” in practice: RSP (the stack pointer) and RBP (the base pointer, sometimes called the frame pointer). RSP always points at the top of the current stack, and instructions like push, pop, call, and ret update it automatically as a side effect. RBP is conventionally used to mark a fixed reference point within a function’s stack frame, so that local variables and arguments can be addressed with a constant offset even as RSP itself moves around during the function’s execution. You can technically use them for scratch arithmetic, but doing so is asking for trouble unless you know exactly what you’re doing and have saved their original values somewhere safe.

3. Basic Assembly Instructions

Basic Syntax

x64 assembly can be written in two different syntaxes, Intel and AT&T, and they look different enough that reading one while expecting the other will genuinely confuse you. This tutorial uses Intel syntax, which is the convention favored by NASM (the assembler we’ll use throughout) and, generally, by Windows tooling. The defining feature of Intel syntax is that the destination comes first: mov rax, rbx means “move the value of RBX into RAX,” not the other way around. AT&T syntax (used by GNU tools like GAS and shown in a lot of Linux disassembly output by default) reverses that order and prefixes registers with a % sign, so keep that in mind if you ever find yourself reading GDB output that looks unfamiliar.

mov rax, rbx  ; Copy RBX into RAX
add rax, 5    ; Add 5 to RAX

Notice the semicolon: everything after it on a line is a comment, ignored entirely by the assembler. Comments matter more in assembly than almost any other language, because the instructions themselves carry so little semantic meaning on their own. mov rax, rbx tells you nothing about why you’re moving that value; a good comment fills in the intent that the mnemonic can’t express.

Common Instructions

The instruction set is enormous (x64 has hundreds of opcodes once you count all the SIMD and specialized extensions), but you can get remarkably far with a small, boring core of arithmetic and data-movement instructions. Here are the ones you’ll reach for constantly:

A couple of these deserve a closer look because they behave less obviously than their names suggest. mul and div don’t work like a typical two-operand instruction where you specify both operands explicitly; instead, they implicitly use RAX (or its smaller aliases) as one operand and stash the result across RAX and RDX. For an unsigned multiply, mul rdi computes RAX * RDI and writes the result as a 128-bit value split across RDX (high 64 bits) and RAX (low 64 bits). div is the mirror image: it divides the 128-bit value in RDX:RAX by the operand, leaving the quotient in RAX and the remainder in RDX. If you forget to clear RDX before a division and it happens to hold garbage from an earlier computation, you’ll get a wrong answer, or worse, a divide-by-zero style fault, that’s maddening to debug if you don’t know this convention exists.

inc and dec look like they’d be equivalent to add reg, 1 and sub reg, 1, and functionally they mostly are, but there’s a subtle difference in how they interact with the CPU’s flags register: inc/dec don’t touch the carry flag, while add/sub do. That matters if you’re chaining these instructions with conditional jumps that depend on the carry flag; using the wrong one can silently break your logic.

Example

section .data
    num db 5

section .text
    global _start

_start:
    mov al, [num]
    inc al

    mov rax, 60
    xor rdi, rdi
    syscall

Walk through this line by line and it tells a complete little story. section .data declares a region of the binary for initialized data, and num db 5 reserves a single byte there, initialized to the value 5. section .text switches to the region that holds actual executable code, and global _start exports the _start label so the linker knows where the operating system should hand off control when the program launches (this is the assembly equivalent of main, except it’s the true entry point, with none of the C runtime setup that normally runs before main gets called).

Inside _start, mov al, [num] loads that byte from memory into the low 8 bits of RAX. The square brackets are doing important work here: num alone would refer to the address of the label, while [num] means “dereference that address and give me the value stored there.” Mixing these up, forgetting the brackets when you meant to load a value, is one of the most common early mistakes in assembly. inc al then adds 1 to that byte, entirely within the register, without ever writing the change back to memory. Finally, the last three lines are the standard Linux exit sequence: put the syscall number for exit (60) into RAX, zero out RDI to signal a clean exit status of 0, and trigger the syscall instruction to hand control to the kernel. We’ll get into exactly what’s happening there in section 7.

4. Memory Addressing

Registers are fast, but they’re also scarce, only sixteen of them, so any real program needs a way to read and write to the much larger pool of memory. x64 gives you a genuinely flexible set of addressing modes, and understanding them is really understanding how the CPU computes an effective address, the actual memory location an instruction ends up touching, from the pieces you give it.

• Direct: mov rax, [0x1000] reads whatever 8 bytes live at the literal, hardcoded memory address 0x1000. This is rare in real code (hardcoded addresses aren’t very useful in a world with virtual memory and address space layout randomization), but it’s the simplest case to reason about.
• Indirect: mov rax, [rbx] treats RBX as a pointer: whatever address happens to be sitting in RBX at the time this instruction runs, that’s where the CPU goes to fetch the value. This is the workhorse pattern for following pointers, walking linked structures, or dereferencing anything whose address you computed earlier and stashed in a register.
• Indexed: mov rax, [rbx+rcx] adds RBX and RCX together to form the effective address before dereferencing it. This shows up constantly when you’re iterating over an array: RBX might hold the base address of the array, and RCX an offset that increases each time through a loop.

What the examples above don’t show, but is worth knowing exists, is that x64 addressing can get considerably richer than this: you can combine a base register, an index register, a scale factor (1, 2, 4, or 8, handy for indexing arrays of different element sizes without needing a separate multiply instruction), and a constant displacement, all in a single instruction, like mov rax, [rbx + rcx*8 + 16]. The CPU computes all of that arithmetic as part of decoding the instruction, essentially for free, which is why compilers lean on these addressing modes so heavily instead of emitting separate add instructions to compute addresses by hand.

5. Control Flow (Loops and Branches)

Straight-line code that always executes the same instructions in the same order isn’t very useful on its own; real programs need to make decisions and repeat work. At the hardware level, there’s no such thing as an if statement or a while loop, those are conveniences your compiler builds out of a much more primitive tool: the conditional jump.

Every arithmetic and comparison instruction updates a special register called the FLAGS register (RFLAGS in 64-bit mode), setting individual bits based on properties of the result, such as whether it was zero, negative, or produced a carry out of the top bit. The cmp instruction is the one you’ll use most often to set these flags deliberately: it subtracts its second operand from its first, throws away the numeric result, and keeps only the flags that subtraction would have produced. Conditional jump instructions then simply check those flags and decide whether to branch.

Conditional Jumps

It’s worth noting that “equal,” “greater,” and “less” here are relative to whatever the last flag-setting instruction computed, not some inherent property of the jump itself. That’s why cmp and the conditional jump that follows it are really a matched pair: cmp rcx, 0 followed by jne .loop reads naturally as “jump to .loop if RCX was not equal to 0,” but under the hood what’s really happening is cmp sets the zero flag based on rcx - 0, and jne checks whether that zero flag came out clear. There are also signed and unsigned variants of the greater/less family (jg/jl for signed comparisons, ja/jb for unsigned), which matters a lot if you’re ever comparing values that could be interpreted either way, using the wrong one is a classic source of bugs when a value you assumed was always positive turns out to have its high bit set.

Example Loop

section .data
    counter db 10

section .text
    global _start

_start:
    mov rcx, [counter]

.loop:
    dec rcx
    cmp rcx, 0
    jne .loop

    mov rax, 60
    xor rdi, rdi
    syscall

This loop counts down from 10 to 0. The label .loop (the leading dot makes it a “local” label in NASM, scoped to the nearest preceding non-local label, which is a handy way to avoid naming collisions between the loop labels of different functions) marks the top of the repeated block. Each pass decrements RCX, compares it against 0, and jumps back to the top if it hasn’t reached 0 yet. Once RCX hits 0, jne falls through instead of branching, and execution continues into the exit sequence. If you’ve written C, this is exactly the kind of loop a while (counter--) or for loop compiles down to, minus all the syntactic sugar.

6. Functions and Calling Conventions

Assembly doesn’t have functions in the way C does; what it has is call and ret, plus a convention, an agreed-upon set of rules about where arguments go, where the return value comes back, and which registers a function is allowed to clobber versus which ones it must preserve. Conventions matter enormously because assembly code, C code, and libraries compiled from other languages all need to interoperate, and none of that works unless everyone agrees on the rules.

call does two things atomically: it pushes the address of the instruction immediately following the call onto the stack (the “return address”), and then jumps to the target label. ret reverses that: it pops the return address off the stack and jumps to it. This pairing is exactly why balancing your stack matters so much in assembly; if a function pushes something onto the stack and forgets to pop it before executing ret, the CPU will pop that leftover value instead of the real return address and jump to a nonsensical location, typically crashing the program (or, in the worst case, becoming an exploitable bug if an attacker can control what ends up on the stack).

Linux x64 Calling Convention

On Linux, the convention that everyone (GCC, Clang, hand-written assembly that wants to interoperate with C) agrees to follow is called the System V AMD64 ABI. It specifies, among many other things, exactly which registers carry the first six integer or pointer arguments to a function:

Any arguments beyond the sixth get pushed onto the stack instead, which is one reason functions with a huge number of parameters are marginally slower to call than ones that fit entirely in registers. The return value, for anything that fits in a single register, comes back in RAX. The ABI also draws a line between “caller-saved” registers (RAX, RCX, RDX, RSI, RDI, R8 through R11), which a called function is free to overwrite without asking permission, and “callee-saved” registers (RBX, RBP, R12 through R15), which a function must restore to their original values before returning if it touches them at all. Get this wrong in hand-written assembly that’s meant to be called from C, and you’ll get baffling bugs where a caller’s local variable mysteriously changes value after a completely unrelated function call, because that function used a callee-saved register as scratch space without saving and restoring it.

Example Function (Factorial)

section .text
    global _start

_start:
    mov rdi, 5
    call factorial

    mov rax, 60
    xor rdi, rdi
    syscall

factorial:
    mov rax, 1
    cmp rdi, 1
    jle .done

.loop:
    mul rdi
    dec rdi
    cmp rdi, 1
    jg .loop

.done:
    ret

This computes 5 factorial using the convention we just covered: the argument (5) goes into RDI before the call, exactly where the ABI says the first argument belongs. Inside factorial, RAX is initialized to 1 (the multiplicative identity, so multiplying by it the first time through doesn’t change anything), and there’s an early-out check: if RDI is already less than or equal to 1, there’s no multiplying left to do, so it jumps straight to .done. Otherwise, the loop repeatedly multiplies RAX by the current value of RDI and decrements RDI, continuing as long as RDI is still greater than 1. Remember from section 3 that mul rdi implicitly multiplies RAX by RDI and leaves the result in RAX (with overflow spilling into RDX, though for small inputs like 5 that never comes into play). When the loop ends, RAX holds the factorial, and ret returns control to right after the call factorial instruction in _start, with the result sitting exactly where the ABI says a caller should look for it.

One thing to notice: this function never touches RSP or pushes anything onto the stack, so it doesn’t need to worry about balancing pushes and pops before its ret. That’s a nice property of simple leaf functions like this one, but the moment a function needs local stack storage or calls other functions itself, you have to be much more deliberate about maintaining stack discipline.

7. System Calls in Assembly

A user-space program can’t directly read a file, write to a socket, or exit the process; those are privileged operations that only the operating system kernel is allowed to perform, because the kernel is responsible for enforcing security boundaries between processes and hardware. Whenever your program needs the OS to do something on its behalf, it makes a system call, and on modern x64 Linux, the mechanism for doing that is the aptly-named syscall instruction. Executing syscall triggers a fast transition into kernel mode, the CPU switches privilege levels, hands control to a kernel-defined entry point, the kernel does whatever work was requested, and then control returns to user space right after the syscall instruction.

The convention for which syscall gets invoked, and what arguments it takes, mirrors the function-calling convention but with its own twist: the syscall number goes in RAX, and the arguments go in RDI, RSI, RDX, R10, R8, and R9 in that order. Notice that the fourth argument uses R10 instead of RCX, unlike the regular calling convention we saw in section 6. That’s not an arbitrary inconsistency, it’s because the syscall instruction itself clobbers RCX (using it internally to save the return address) and R11 (using it to save flags), so the kernel ABI had to route the fourth argument through a different register to avoid stepping on itself.

• Use the syscall instruction to make the request.
• Arguments go in RDI, RSI, RDX, R10, R8, R9.
• The syscall number goes in RAX, and after the call returns, RAX holds the result (or a negative errno value on failure).

These numbers aren’t universal constants baked into the CPU, they’re specific to the Linux kernel’s syscall table on x64 (other operating systems, and even Linux on other architectures, number their syscalls differently). If you ever need the full, authoritative list, it’s worth knowing that the kernel source ships a table mapping every syscall name to its number for each architecture.

Example (Writing to stdout)

section .data
    msg db 'Hello, World!', 0xA
    len equ $-msg

section .text
    global _start

_start:
    mov rax, 1
    mov rdi, 1
    mov rsi, msg
    mov rdx, len
    syscall

    mov rax, 60
    xor rdi, rdi
    syscall

This is the classic “hello world,” but written at a level where every byte is visible. msg db 'Hello, World!', 0xA defines the string in memory, followed explicitly by 0xA, the newline character, since assembly string literals don’t automatically get one appended the way a C printf format string implicitly handles newlines you write yourself. len equ $-msg computes the string’s length at assembly time: $ refers to the current address (right after the string bytes), so subtracting the address of msg from it gives you the exact byte count, computed once by the assembler rather than something you’d have to count and hardcode by hand (and risk getting wrong if the string ever changes).

The syscall itself follows the pattern from the table above: RAX gets 1 (the write syscall number), RDI gets 1 (the file descriptor for stdout; 0 is stdin and 2 is stderr, a convention that goes back decades in Unix), RSI gets the address of the buffer to write, and RDX gets the number of bytes to write. After syscall returns, RAX would hold the number of bytes actually written (write can partially succeed, though for a short, simple write to a terminal that’s rarely something you need to handle), but this example doesn’t check it before moving straight into the exit sequence.

8. Data Types and Structures

Assembly doesn’t have a type system in the way C or any higher-level language does; the CPU doesn’t know or care whether the bytes at a given address represent an integer, a character, or a fragment of a floating-point number, it just moves and operates on bits of a specified width. What NASM’s data-definition directives give you is really just a way to tell the assembler how many bytes to reserve and how to initialize them, and it’s on you to keep track of what those bytes are supposed to mean.

The names here are a direct callback to the register aliasing table back in section 2: a “word” is 16 bits because that’s the native register width of the original 16-bit 8086, and everything since has been named relative to that historical baseline, which is why a 32-bit value is a “double word” and a 64-bit value is a “quad word” rather than something more intuitively named. You’ll also see resb, resw, resd, and resq in real code, which reserve uninitialized space of the same widths instead of writing an initial value, useful for buffers you plan to fill in at runtime rather than at assembly time.

9. Interfacing Assembly with C

One of the most practical reasons to learn assembly at all, beyond pure curiosity about how your computer works, is that it lets you drop down to hand-written machine-level code from within an otherwise normal C project, whether that’s for a tight inner loop that needs every cycle you can squeeze out of it, or to use a CPU instruction that doesn’t have a convenient C intrinsic. Because both NASM-assembled object files and GCC-compiled object files ultimately produce standard ELF object files that follow the same System V ABI we covered in section 6, they can be linked together as if they’d always been part of the same program.

Assembly Function

section .text
    global sum

sum:
    mov rax, rdi
    add rax, rsi
    ret

This function follows the calling convention to the letter: it expects its two arguments in RDI and RSI (exactly where a C compiler will place them when it generates a call to sum), computes their sum, and leaves the result in RAX, exactly where the caller will look for it. The global sum directive is what makes this label visible to the linker outside this object file; without it, sum would be local to this assembly file and the linker would have no way to resolve a reference to it from C.

C Program

#include <stdio.h>

extern long sum(long a, long b);

int main() {
    long result = sum(5, 3);
    printf("Result: %ld\n", result);
    return 0;
}

The extern declaration is doing the real work of bridging the two languages: it tells the C compiler “trust me, a function called sum with this signature exists somewhere, just emit a call to it and let the linker sort out where it actually lives.” As far as the C code is concerned, this is indistinguishable from calling any other C function, because the ABI guarantees that the mechanics of argument passing and return values are identical either way. That’s really the whole point of a calling convention: it lets code written in completely different languages, by different compilers, at different times, cooperate as long as everyone honors the same contract.

Compile

Building this requires two separate steps, since NASM and GCC are handling different files. First, assemble the .asm file into an object file:

nasm -f elf64 -o sum.o sum.asm

The -f elf64 flag tells NASM to produce a 64-bit ELF object file, the same format GCC produces on Linux, which is what makes the next step possible. Then compile and link the C source together with that object file in a single invocation:

gcc -o main main.c sum.o

GCC compiles main.c into its own object file internally, and then hands everything off to the linker, which resolves the sum symbol referenced in main.c against the sum symbol defined in sum.o, and stitches the two into a single executable. Run the resulting main binary and you’ll see Result: 8 printed, computed entirely by hand-written assembly, called seamlessly from ordinary C.