Skip to content

Array Backends

SGN-TS follows one principle: the array is the backend. A buffer's backend (NumPy, PyTorch, …) is read from its data, not configured on an element. Feed a transform NumPy arrays and it runs in NumPy; feed it PyTorch tensors and it runs in PyTorch. There is no backend setting threaded through a pipeline, and no implicit conversions — the backend changes only where you put a Converter.

Two words recur and are worth pinning down: a namespace is the xp object you compute with (array_namespace(data)); a backend is its name — the string "numpy"/"torch" an element declares in backends and the framework checks. Same underlying thing (NumPy vs PyTorch), seen as the object you call versus the label you compare.

Writing backend-agnostic element code

Element code that does standard array work uses the Array API namespace (xp), obtained from the data itself:

import numpy

from sgnts.base import array_namespace, new_zeros

data = numpy.ones((2, 1024))   # ... or a torch tensor
xp = array_namespace(data)     # the standard namespace for this array
total = xp.sum(data)           # same call for numpy or torch

# Create new arrays that match existing data (same namespace, dtype, device).
# Use this instead of numpy.zeros / torch.zeros so gaps and scratch buffers
# follow the stream's backend:
z = new_zeros(data, (2, 1024))

xp covers the standard creation/manipulation ops (zeros, concat, stack, matmul, sum, arange, …). A few operations are not in the standard for every backend (notably pad, and convolution/correlation). Those live in one named place, sgnts.base.signal, as namespace-dispatched helpers:

from sgnts.base.signal import pad

padded = pad(data, (8, 8))     # zero-pads the last axis; works on numpy or torch

Declaring what an element supports

Most operations are uniform across backends, but some elements are genuinely backend-specific (a NumPy-only transform using SciPy, a Torch-only one using conv1d). Each transform declares its capability with a backends attribute, which the framework validates against the actual data at runtime:

from sgnts.base import ANY_BACKEND, TSTransform


class MyUniversal(TSTransform):
    backends = ANY_BACKEND                       # pure xp; any backend


class MyMultiBackend(TSTransform):
    backends = frozenset({"numpy", "torch"})     # dispatch internally per backend


class MyNumpyOnly(TSTransform):
    backends = frozenset({"numpy"})              # torch input -> BackendError

The default is frozenset({"numpy"}), so a numpy-only element needs no declaration. Feeding an element data in a backend it does not declare raises a clear BackendError naming the pad and telling you to insert a Converter on that link. A single-backend element can then just import its library directly (import numpy / import torch) — the framework guarantees the input matches.

Declaring an element's output backend and dtype

Real output data describes itself — feed a transform float32 and it emits float32, because the array it produced is a float32 array. You only need to say something when your all-gap output frames (which carry no array) would otherwise guess wrong — i.e. when your output dtype differs from your input. That declaration is a single hook, output_prototype(self, pad), which returns a zero-length example of what the pad emits; the framework reads namespace, device and dtype off it.

Most transforms override nothing — the default inherits the input's backend and dtype unchanged (a filter, a resampler, a gain that stays real):

class HighPass(TSTransform):
    def process(self, in_frame, out_frame): ...
    # no output_prototype -> output spec = input spec

When the output dtype does differ, build the prototype from the inputs with the input_prototype helper and let the operation decide the dtype:

class Amplify(TSTransform):
    factor: float = 1

    def output_prototype(self, pad):
        # output dtype = result_type(input, factor): a float64 factor on a float32
        # input promotes to float64; promotion is whatever the backend does
        return self.input_prototype("in") * self.factor

input_prototype(name, dtype=None, device=None) hands you a zero-length array in that input's namespace and device, so namespace and device are inherited from the data — you never restate them, and you cannot accidentally change them. A transform may change only dtype; the framework asserts your prototype keeps the input's namespace+device (device changes only at a Converter). More shapes:

# fixed dtype change (complex -> real):
def output_prototype(self, pad):
    return abs(self.input_prototype(self.sink_pad_names[0]))

# many inputs -> one output, promoted across them:
def output_prototype(self, pad):
    return sum(self.input_prototype(p) for p in self.sink_pad_names)

# many outputs, each its own dtype (M:N): branch on the output pad
def output_prototype(self, pad):
    if pad.pad_name == "snr":
        return self.input_prototype("strain") * self.input_prototype("template")
    if pad.pad_name == "power":
        return abs(self.input_prototype("strain")) ** 2

Honest caveat. An operation that changes both shape and dtype (an FFT, real→complex) can't be run on a length-0 array — declare the dtype instead of running the op:

def output_prototype(self, pad):
    return self.input_prototype(self.sink_pad_names[0], dtype=complex)   # FFT -> complex

That's the only case where the prototype isn't literally "your operation on a tiny input." A dtype-preserving shape change (convolution, resampling) needs no override at all.

Materializing gaps

A gap buffer carries no array. To get a dense array for a frame — gaps filled with zeros in the frame's backend — use TSFrame.filleddata():

signal = frame.filleddata()    # gaps -> zeros in the frame's backend/dtype/device

This is the only backend-safe way to materialize a frame's gaps; the frame knows its backend even when it is entirely gaps. To fill a single bare buffer, pass a reference array whose backend the gap zeros should match — buf.filleddata(reference).

Choosing and changing the backend

The backend is chosen only at the edges of a stream:

Sources produce the initial arrays, so a source's backend is simply the backend of the data it creates — the bundled FakeSeriesSource produces NumPy (with an optional dtype). A source declares its output with the same output_prototype hook as a transform, but returns a literal (it has no input to inherit from). A data-first source needs nothing; one whose first frame may be a gap declares the backend so the gap is never silently NumPy:

import torch
from sgnts.base import TSSource


class MyTorchSource(TSSource):
    def output_prototype(self, pad):
        # a zero-length prototype in the backend/device/dtype this pad emits
        return torch.zeros(0, dtype=torch.float16, device="cuda")

Omitting this on a gap-first, non-NumPy source is a hard error when the first real array arrives — never a silent fall back to NumPy.

Converter is the one element that may change backend (and device/dtype) mid-pipeline. It declares changes_backend = True, which is what lifts the namespace/device guardrail for it and it alone. It is explicit and visible in the pipeline definition:

from sgnts.transforms import Converter

# numpy -> torch, on the GPU at float16
conv = Converter(name="to_torch", backend="torch", dtype="float16", device="cuda")

If a numpy stream reaches a torch-only element with no Converter in between, you get a BackendError, never a silent copy.

Device and precision

Device and dtype travel with the data, chosen at the source or a Converter — the same way the backend does. Running on CPU vs CUDA, or at fp16 vs fp32, is just what the edge selected; the pipeline body is identical. Per-(dtype, device) state (e.g. a resampler's kernel) adapts to the data, so one element instance can serve fp16 and fp32 streams at once.