> For the complete documentation index, see [llms.txt](https://healdocs.lattica.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://healdocs.lattica.ai/core-concepts-explained/tensor.md). # Tensor Tensors are the primary data structure in HEAL. They represent multi-dimensional arrays with fixed shape and data type. Every function in the HEAL runtime operates over tensors, which are passed as pointers into device memory. {% hint style="info" %} We adopt a widely accepted approach to tensor structure and semantics, consistent with **NumPy** and other modern ML frameworks like **PyTorch**. {% endhint %} This page introduces how tensors are structured in HEAL: their **data types**, **shapes**, **strides**, and **broadcasting** behavior. ## Data Types HEAL tensors support integer types (`int32_t` ;`int64_t`) Every tensor must have a clearly defined scalar type. The type is determined at tensor creation and cannot be changed. **Example:** ```cpp std::shared_ptr> my_tensor; ``` This declares a tensor of 32-bit integers, managed on the device. ## Shape A tensor’s **shape** defines its dimensionality - the size of the tensor along each axis. **Example:** A tensor with shape `[2, 3]` contains 2 rows and 3 columns: ``` [ [1, 2, 3], [4, 5, 6] ] ``` * Shape is immutable unless explicitly changed (e.g. via `reshape`). * Shape compatibility is enforced in all HEAL operations. ## Strides **Strides** describe how tensor elements are laid out in memory. Each stride tells you how many memory steps to take to move between elements along a particular axis. Strides are especially important when tensors are **non-contiguous**, such as after slicing, reshaping, or transposing. **Example:** A tensor with shape `[2, 3]` laid out in row-major order will have strides `[3, 1]`: * Move 3 units in memory to go from row 0 to row 1. * Move 1 unit to move across columns. Strides allow for memory-efficient views and slicing without copying data. ## Broadcasting Semantics Broadcasting allows element-wise operations between tensors of different shapes, as long as the shapes are *broadcastable*. This enables operations without explicitly reshaping tensors. ### Broadcasting Rules Two tensors are **broadcastable** if: 1. Both have at least one dimension. 2. Starting from the **last (trailing)** dimension and comparing backward: * Dimensions are equal **or** * One of the dimensions is `1` **or** * One of the tensors has **fewer dimensions**, in which case it's **left-padded with 1s** to match the other tensor's rank.
Example ```plaintext A.shape = [ 3, 1, 5] B.shape = [2, 1, 3, 1, 1] ``` Becomes (after left-padding `A` with two 1s): ```plaintext A.shape = [1, 1, 3, 1, 5] B.shape = [2, 1, 3, 1, 1] ``` Resulting broadcasted shape: ```plaintext [2, 1, 3, 1, 5] ```
If these rules hold for every dimension, the shapes are broadcastable. The resulting shape is the maximum size in each dimension. #### ✓ Valid Broadcasting Example ``` Shape A: [5, 1, 4, 1] Shape B: [3, 1, 1] Resulting shape: [5, 3, 4, 1] ```
Why is this a valid example? To align shapes, we left-pad B’s shape: ``` Shape A: [5, 1, 4, 1] Shape B: [1, 3, 1, 1] ``` Comparison (from last to first): | Dimension | A | B | Valid? | | --------- | - | - | ---------- | | 4th | 1 | 1 | ✓ equal | | 3rd | 4 | 1 | ✓ one is 1 | | 2nd | 1 | 3 | ✓ one is 1 | | 1st | 5 | 1 | ✓ one is 1 | ✓ All dimensions are compatible → **broadcasting is allowed**
#### ✘ Invalid Broadcasting Example ``` Shape A: [5, 2, 4, 1] Shape B: [3, 1, 1] // Error: dim mismatch (2 ≠ 3) in position 2 ```
Why is this an invalid example? Left-padding B’s shape: ``` Shape A: [5, 2, 4, 1] Shape B: [1, 3, 1, 1] ``` Comparison: | Dimension | A | B | Valid? | | --------- | - | - | ---------- | | 4th | 1 | 1 | ✓ equal | | 3rd | 4 | 1 | ✓ one is 1 | | 2nd | 2 | 3 | ✘ mismatch | | 1st | 5 | 1 | ✓ one is 1 | ✘ Dimension 2: `2 ≠ 3` and neither is 1 → **broadcasting not allowed**
### Additional Note In-place operations (where input and output refer to the same tensor) are **only allowed if broadcasting does not change the input's shape**. If a tensor needs to be **broadcast to a larger shape**, it cannot safely be used as the output - doing so would write outside its allocated memory. **Example:** ```plaintext A.shape = [4, 1] B.shape = [4, 3] C.shape = [4, 3] modsum_ttc(A, B, 6, A); // ❌ Error: A would be broadcast to [4, 3] modsum_ttc(A, B, 6, B); // ✔ Safe: A and B shape remains unchanged modsum_ttc(A, B, 6, C); // ✔ Safe: A, B and C shape remains unchanged ``` *** {% hint style="info" %} Want to understand how tensors are transferred between host and accelerator? See [**HEAL Memory and Execution Model**](/core-concepts-explained/memory-and-execution-model.md) for the full memory model and lifecycle. {% endhint %} ***