xmake-docs/docs/api/scripts/extension-modules/core/base/bytes.md

bytes

The bytes module provides binary data buffer operations for handling raw byte data. This is an extension module of xmake.

::: tip TIP To use this module, you need to import it first: import("core.base.bytes") :::

bytes

• Create a byte buffer

  import("core.base.bytes")
  
  local buff = bytes(size)
  

The bytes constructor supports multiple creation methods, providing flexible buffer creation and management.

Create a buffer of specified size

-- Create a 1024-byte buffer
local buff = bytes(1024)
print("Buffer size:", buff:size())  -- Output: 1024

-- Create buffer and initialize with specified value
local buff = bytes(100, 0)   -- Initialize to 0
local buff = bytes(100, 255) -- Initialize to 255
local buff = bytes(100, 'A') -- Initialize to character 'A'

Create from string

Create a bytes object from a string, commonly used to convert strings to binary data:

local buff = bytes("hello world")
print(buff:size())  -- Output: 11
print(buff:str())   -- Output: hello world

::: warning WARNING bytes objects created from strings are read-only and cannot be modified. :::

Create a slice

Create a slice from an existing bytes object, sharing underlying memory without copying data:

local original = bytes("123456789")
local slice = bytes(original, 3, 5)  -- Slice bytes 3-5
print(slice:str())  -- Output: 345

Concatenate multiple buffers

-- Concatenate using parameter list
local buff = bytes(bytes("123"), bytes("456"), bytes("789"))
print(buff:str())  -- Output: 123456789

-- Concatenate using array
local buff = bytes({bytes("123"), bytes("456"), bytes("789")})
print(buff:str())  -- Output: 123456789

Create empty buffer

local buff = bytes()   -- Empty buffer
local buff = bytes({}) -- Empty buffer

Index operations

bytes objects support accessing and modifying individual bytes through indexing (indices start from 1):

local buff = bytes(9)
buff:copy("123456789")

-- Read byte
local byte_value = buff[1]
print(byte_value)  -- Output: 49 (ASCII code for '1')

-- Modify byte
buff[1] = string.byte('2')
print(buff:str())  -- Output: 223456789

Access slices through range indexing:

local buff = bytes("123456789")
local slice = buff[{1, 4}]
print(slice:str())  -- Output: 1234

-- Range assignment
local buff = bytes(9)
buff[{1, 9}] = bytes("123456789")
print(buff:str())  -- Output: 123456789

Concatenation operation

Use the .. operator to concatenate two bytes objects, creating a new buffer:

local buff1 = bytes("123")
local buff2 = bytes("456")
local buff3 = buff1 .. buff2
print(buff3:str())  -- Output: 123456

bytes:size

• Get buffer size

  local size = buff:size()
  

Returns the number of bytes in the buffer.

local buff = bytes(1024)
print(buff:size())  -- Output: 1024

bytes:str

• Convert to string

  local str = buff:str()
  local str = buff:str(start, last)
  

Converts a bytes object to a string, optionally specifying the conversion range.

Parameters:

• start (optional): Start position, default 1
• last (optional): End position, default is buffer size

Commonly used to convert data read from network or files to strings:

import("core.base.bytes")

local buff = bytes("hello world")
print(buff:str())        -- Output: hello world
print(buff:str(1, 5))    -- Output: hello
print(buff:str(7))       -- Output: world

bytes:slice

• Create a slice

  local slice = buff:slice(start, last)
  

Creates a slice of the buffer, returns a new bytes object that shares the underlying memory (no data copying).

Parameters:

• start: Start position
• last: End position

Slicing is an efficient data access method that avoids data copying:

local buff = bytes("123456789")
local slice = buff:slice(3, 5)
print(slice:str())  -- Output: 345
print(slice:size()) -- Output: 3

bytes:clone

• Clone the buffer

  local new_buff = buff:clone()
  

Creates a complete copy of the buffer, allocating new memory and copying all data.

Unlike slice, clone copies data, so they don't affect each other:

local original = bytes("hello")
local cloned = original:clone()
print(cloned:str())  -- Output: hello

-- Modifying cloned buffer doesn't affect original
cloned[1] = string.byte('H')
print(cloned:str())    -- Output: Hello
print(original:str())  -- Output: hello (unchanged)

bytes:copy

• Copy data to buffer

  buff:copy(src, start, last)
  

Copies data from source to the beginning of the buffer.

Parameters:

• src: Source data, can be string or bytes object
• start (optional): Source data start position, default 1

• last (optional): Source data end position, default is source data size

  local buff = bytes(9)
  buff:copy("123456789")
  print(buff:str())  -- Output: 123456789
  
  -- Copy only partial data
  local buff = bytes(5)
  buff:copy("123456789", 5, 9)
  print(buff:str())  -- Output: 56789
  

bytes:copy2

• Copy data to specified position

  buff:copy2(pos, src, start, last)
  

Copies data from source to the specified position in the buffer.

Parameters:

• pos: Target position
• src: Source data, can be string or bytes object
• start (optional): Source data start position
• last (optional): Source data end position

Used to concatenate multiple segments of data in a buffer:

local buff = bytes(18)
buff:copy("123456789")       -- Copy to beginning
buff:copy2(10, "123456789")  -- Copy to position 10
print(buff:str())  -- Output: 123456789123456789

bytes:move

• Move data to buffer beginning

  buff:move(start, last)
  

Moves the specified range of data within the buffer to the beginning, supporting memory overlap-safe movement.

Parameters:

• start: Source data start position

• last (optional): Source data end position

  local buff = bytes(9):copy("123456789")
  buff:move(5, 9)
  print(buff:str())  -- Output: 567896789 (5-9 moved to beginning)
  

bytes:move2

• Move data to specified position

  buff:move2(pos, start, last)
  

Moves the specified range of data within the buffer to the specified position.

Parameters:

• pos: Target position
• start: Source data start position

• last (optional): Source data end position

  local buff = bytes(9):copy("123456789")
  buff:move2(2, 5, 9)
  print(buff:str())  -- Output: 156789789
  

bytes:u8

• Read unsigned 8-bit integer

  local value = buff:u8(offset)
  

Reads a byte from the specified offset as an unsigned 8-bit integer (0-255).

bytes:u8_set

• Write unsigned 8-bit integer

  buff:u8_set(offset, value)
  

Writes an unsigned 8-bit integer value to the specified offset.

local buff = bytes(10)
buff:u8_set(1, 255)
local value = buff:u8(1)
print(value)  -- Output: 255

bytes:s8

• Read signed 8-bit integer

  local value = buff:s8(offset)
  

Reads a byte from the specified offset as a signed 8-bit integer (-128 to 127).

bytes:u16le

• Read unsigned 16-bit integer (little-endian)

  local value = buff:u16le(offset)
  

Reads 2 bytes from the specified offset as an unsigned 16-bit integer (little-endian byte order).

bytes:u16le_set

• Write unsigned 16-bit integer (little-endian)

  buff:u16le_set(offset, value)
  

Writes an unsigned 16-bit integer to the specified offset (little-endian byte order).

local buff = bytes(10)
buff:u16le_set(5, 12346)
local value = buff:u16le(5)
print(value)  -- Output: 12346

bytes:u16be

• Read unsigned 16-bit integer (big-endian)

  local value = buff:u16be(offset)
  

Reads 2 bytes from the specified offset as an unsigned 16-bit integer (big-endian byte order).

bytes:u16be_set

• Write unsigned 16-bit integer (big-endian)

  buff:u16be_set(offset, value)
  

Writes an unsigned 16-bit integer to the specified offset (big-endian byte order).

bytes:u32le

• Read unsigned 32-bit integer (little-endian)

  local value = buff:u32le(offset)
  

Reads 4 bytes from the specified offset as an unsigned 32-bit integer (little-endian byte order).

bytes:u32le_set

• Write unsigned 32-bit integer (little-endian)

  buff:u32le_set(offset, value)
  

Writes an unsigned 32-bit integer to the specified offset (little-endian byte order).

Used for handling binary protocols and data formats:

local buff = bytes(20)

-- Write protocol header (little-endian format)
buff:u16le_set(1, 0x1234)  -- Magic number
buff:u32le_set(3, 100)      -- Data length

-- Read protocol header
local magic = buff:u16le(1)
local length = buff:u32le(3)
print(string.format("Magic: 0x%04X, Length: %d", magic, length))

bytes:u32be

• Read unsigned 32-bit integer (big-endian)

  local value = buff:u32be(offset)
  

Reads 4 bytes from the specified offset as an unsigned 32-bit integer (big-endian byte order).

bytes:u32be_set

• Write unsigned 32-bit integer (big-endian)

  buff:u32be_set(offset, value)
  

Writes an unsigned 32-bit integer to the specified offset (big-endian byte order).

bytes:dump

• Display buffer contents in hexadecimal format

  buff:dump()
  buff:dump(start, last)
  

Displays buffer contents in hexadecimal and ASCII format, similar to the hexdump tool.

Parameters:

• start (optional): Start position
• last (optional): End position

Used for debugging binary data, visually displaying memory contents:

local buff = bytes("hello world, this is a test")
buff:dump()
-- Output similar to:
-- 00000000  68 65 6C 6C 6F 20 77 6F  72 6C 64 2C 20 74 68 69  hello world, thi
-- 00000010  73 20 69 73 20 61 20 74  65 73 74                 s is a test

bytes:readonly

• Check if buffer is read-only

  local is_readonly = buff:readonly()
  

Returns true if the buffer is read-only and cannot be modified. bytes objects created from strings are read-only.

local buff1 = bytes("hello")
print(buff1:readonly())  -- Output: true (cannot modify)

local buff2 = bytes(10)
print(buff2:readonly())  -- Output: false (can modify)

-- Attempting to modify read-only buffer will raise error
local buff = bytes("test")
-- buff[1] = 65  -- Error! Read-only buffer cannot be modified

::: tip TIP bytes objects can be seamlessly used across socket, pipe, lz4, and other modules. Pre-creating large buffers and reusing them can reduce memory allocation overhead. :::

::: warning WARNING

• Indices start from 1 (following Lua convention)
• bytes objects created from strings are read-only
• Slices share memory with the original bytes object
• Pay attention to byte order (big-endian/little-endian) when using integer read/write interfaces :::