This document provides a detailed API reference for the cTensor library, a lightweight tensor library for C with automatic differentiation.
- Core Data Structures
- Library Initialization & Finalization
- TensorShape Utilities
- Tensor Creation & Management
- Tensor Operations
- Neural Network Functions
- Automatic Differentiation
- Optimizers
- Gradient Clipping
- Memory Management
- Utilities & Miscellaneous
These are the fundamental data types used throughout the cTensor library.
A type definition for tensor shapes, supporting up to 4 dimensions.
typedef int TensorShape[4];A structure storing the raw tensor data.
typedef struct FloatBuffer {
int numel; /**< Number of elements in the buffer */
float flex[]; /**< Flexible array member containing the actual data */
} FloatBuffer;The main tensor structure, containing its shape, data, and a node for gradient computation.
typedef struct Tensor {
TensorShape shape; /**< Tensor dimensions [dim0, dim1, dim2, dim3] */
FloatBuffer* data; /**< Pointer to data buffer */
GradNode* node; /**< Gradient computation node (NULL if no gradients) */
} Tensor;A node in the computation graph used for automatic differentiation.
typedef struct GradNode {
struct Tensor grad;
struct Tensor (*grad_fn)(struct Tensor self, int i);
struct Tensor inputs[4];
int n_inputs;
const char* name;
int params[4];
} GradNode;Fields:
grad: The accumulated gradient for the tensor associated with this node.grad_fn: A function pointer to the gradient function used in backpropagation.inputs: An array of input tensors that produced the current tensor.n_inputs: The number of input tensors.name: The name of the operation for debugging.params: Additional integer parameters required by the operation.
A structure to hold the results of max or min operations along a dimension.
typedef struct {
Tensor values; /**< Maximum/minimum values */
Tensor indices; /**< Indices of maximum/minimum values */
} TensorMaxMinResult;Initializes the CTensor library and its internal memory management system. Must be called before any other CTensor function.
void cten_initilize();Frees all allocated memory and cleans up internal library structures. Should be called when finished using CTensor.
void cten_finalize();Functions for working with TensorShape types.
Calculates the total number of elements in a tensor shape (product of dimensions).
int TensorShape_numel(TensorShape shape);Gets the number of dimensions in a tensor shape (number of non-zero dimensions).
int TensorShape_dim(TensorShape shape);Normalizes a dimension index to handle negative indices (e.g., -1 for the last dimension).
int TensorShape_asdim(TensorShape shape, int dim);Converts a tensor shape to its string representation.
int TensorShape_tostring(TensorShape shape, char* buf, int size);Functions for creating and manipulating Tensor objects.
Creates a new tensor with uninitialized data.
Tensor Tensor_new(TensorShape shape, bool requires_grad);Creates a new tensor filled with zeros.
Tensor Tensor_zeros(TensorShape shape, bool requires_grad);Creates a new tensor filled with ones.
Tensor Tensor_ones(TensorShape shape, bool requires_grad);Detaches a tensor from the computation graph. The new tensor shares the same data but does not require gradients.
Tensor Tensor_detach(Tensor self);Adds a singleton dimension (a dimension of size 1) at a specified position.
Tensor Tensor_unsqueeze(Tensor self, int dim);Gets the element value at the specified indices.
float Tensor_get(Tensor self, int i, int j, int k, int l);Sets the element value at the specified indices.
void Tensor_set(Tensor self, int i, int j, int k, int l, float value);Prints the contents of a tensor to stdout.
void Tensor_print(Tensor self);These functions perform element-wise arithmetic. They support broadcasting to handle operands with different but compatible shapes.
| Function | Description |
|---|---|
Tensor_add(a, b) |
Adds two tensors. |
Tensor_sub(a, b) |
Subtracts tensor b from a. |
Tensor_mul(a, b) |
Multiplies two tensors. |
Tensor_div(a, b) |
Divides tensor a by b. |
Tensor_pow(a, b) |
Raises tensor a to the power of b. |
Tensor_addf(a, s) |
Adds a scalar s to a tensor. |
Tensor_subf(a, s) |
Subtracts a scalar s from a tensor. |
Tensor_mulf(a, s) |
Multiplies a tensor by a scalar s. |
Tensor_divf(a, s) |
Divides a tensor by a scalar s. |
Tensor_powf(a, s) |
Raises a tensor to the power of a scalar s. |
Performs matrix multiplication of two tensors.
Tensor Tensor_matmul(Tensor self, Tensor other);Transposes a 2D tensor.
Tensor Tensor_transpose(Tensor self);Performs element-wise negation (-self).
Tensor Tensor_neg(Tensor self);Computes the element-wise absolute value (|self|).
Tensor Tensor_abs(Tensor self);Computes the element-wise square (self^2).
Tensor Tensor_square(Tensor self);Computes the element-wise reciprocal (1/self).
Tensor Tensor_reciprocal(Tensor self);These operations reduce a tensor to a single value or along a specified dimension. They are exposed via macros for a simpler API.
Usage:
// Sum of all elements (returns a scalar tensor)
Tensor sum_all = Tensor_sum(my_tensor);
// Sum along dimension 1 (returns a tensor with the dimension removed)
Tensor sum_dim = Tensor_sum(my_tensor, 1);Usage:
// Mean of all elements
Tensor mean_all = Tensor_mean(my_tensor);
// Mean along dimension 1
Tensor mean_dim = Tensor_mean(my_tensor, 1);Usage:
// Max of all elements
Tensor max_val = Tensor_max(my_tensor);
// Max along dimension 1 (returns values and indices)
TensorMaxMinResult max_res = Tensor_max(my_tensor, 1);
Tensor max_vals = max_res.values;
Tensor max_indices = max_res.indices;Underlying Functions: Tensor_max_all(Tensor self), TensorMaxMinResult Tensor_max_dim(Tensor self, int dim)
Usage:
// Min of all elements
Tensor min_val = Tensor_min(my_tensor);
// Min along dimension 1 (returns values and indices)
TensorMaxMinResult min_res = Tensor_min(my_tensor, 1);Underlying Functions: Tensor_min_all(Tensor self), TensorMaxMinResult Tensor_min_dim(Tensor self, int dim)
Finds the indices of the maximum values along the last dimension.
void Tensor_argmax(Tensor self, int* out);Applies a linear transformation (input @ weight + bias).
Tensor nn_linear(Tensor input, Tensor weight, Tensor bias);Initializes a tensor with weights sampled from a Glorot (Xavier) uniform distribution.
Tensor Glorot_init(TensorShape shape, bool requires_grad);| Function | Description |
|---|---|
nn_relu(input) |
Rectified Linear Unit: max(0, input). |
nn_sigmoid(input) |
Sigmoid: 1 / (1 + exp(-input)). |
nn_tanh(input) |
Hyperbolic Tangent. |
nn_elu(self, alpha) |
Exponential Linear Unit. |
nn_selu(self) |
Scaled Exponential Linear Unit. |
nn_softmax(input, dim) |
Softmax function along a specified dimension. |
Computes the cross-entropy loss between true labels and predicted probabilities.
Tensor nn_crossentropy(Tensor y_true, Tensor y_pred);A numerically stable combination of Softmax and Cross-Entropy loss.
Tensor nn_softmax_crossentropy(Tensor y_true, Tensor logits);Computes the Mean Squared Error loss.
Tensor nn_mse_loss(Tensor y_true, Tensor y_pred);Computes the Mean Absolute Error loss.
Tensor nn_mae_loss(Tensor y_true, Tensor y_pred);Computes the Huber loss (a smooth L1 loss).
Tensor nn_huber_loss(Tensor y_true, Tensor y_pred, float delta);| Function | Description |
|---|---|
nn_log(self) |
Element-wise natural logarithm. |
nn_exp(self) |
Element-wise exponential function (e^x). |
nn_sin(self) |
Element-wise sine. |
nn_cos(self) |
Element-wise cosine. |
nn_tan(self) |
Element-wise tangent. |
Performs the backward pass (backpropagation) from this tensor, computing gradients for all tensors in its computation graph that have requires_grad=true.
void Tensor_backward(Tensor self, Tensor grad);Parameters:
self: The tensor to start the backpropagation from (often the final loss).grad: The initial gradient to propagate. For a scalar loss, this is typically a tensor containing the value1.0.
Applies a function to all tensors visited during a backward pass.
int Tensor_backward_apply(Tensor self, void (*f)(Tensor, void*), void* ctx);// Create a new SGD optimizer
optim_sgd* optim_sgd_new(int n_params, Tensor* params, float weight_decay);
// Configure learning rate and momentum
void optim_sgd_config(optim_sgd* self, float lr, float momentum);
// Zero out the gradients of all managed parameters
void optim_sgd_zerograd(optim_sgd* self);
// Perform one optimization step
void optim_sgd_step(optim_sgd* self);optim_adagrad* optim_adagrad_new(int n_params, Tensor* params, float lr, float ε, float weight_decay);
void optim_adagrad_zerograd(optim_adagrad* self);
void optim_adagrad_step(optim_adagrad* self);optim_rmsprop* optim_rmsprop_new(int n_params, Tensor* params, float lr, float β, float ε, float weight_decay);
void optim_rmsprop_zerograd(optim_rmsprop* self);
void optim_rmsprop_step(optim_rmsprop* self);optim_adam* optim_adam_new(int n_params, Tensor* params, float lr, float β1, float β2, float ε, float weight_decay);
void optim_adam_zerograd(optim_adam* self);
void optim_adam_step(optim_adam* self);Functions to prevent exploding gradients during training.
Clips the gradients of a set of parameters by their global L2 norm.
void cten_clip_grad_norm(Tensor* params, int n_params, float max_norm);Clips gradients element-wise to a maximum absolute value.
void cten_clip_grad_value(Tensor* params, int n_params, float max_value);Clips gradients element-wise to be within [min_value, max_value].
void cten_clip_grad_value_range(Tensor* params, int n_params, float min_value, float max_value);cTensor uses a pool-based memory allocator to manage tensor memory, which is especially useful for controlling memory usage during different phases like training epochs.
Begins a new memory allocation pool. All subsequent tensor allocations will be associated with this pool ID.
void cten_begin_malloc(PoolId id);Ends the current memory allocation pool, returning to the previous one in the stack.
void cten_end_malloc();Frees all tensors that were allocated in the specified pool.
void cten_free(PoolId id);Disables gradient computation globally, useful for inference or validation.
Enters evaluation mode.
void cten_begin_eval();Checks if the library is currently in evaluation mode.
bool cten_is_eval();Exits evaluation mode, re-enabling gradient computation.
void cten_end_eval();Loads the built-in Iris dataset.
int load_iris_dataset(const float (**X)[4], const int** y);Normalizes a dataset using the mean and standard deviation from its training split.
void Tensor_normalize_dataset(const float (*X)[4], float (*X_norm)[4], int n_samples, int n_train_samples, int n_features);Randomly shuffles a dataset (features and labels together).
void Tensor_shuffle_dataset(const float (*X)[4], const int* y, float (*X_shuffled)[4], int* y_shuffled, int n_samples, int n_features);Asserts that a condition is true, otherwise prints a formatted error message and exits.
void cten_assert(bool cond, const char* fmt, ...);Asserts that two tensor shapes are equal.
void cten_assert_shape(const char* title, TensorShape a, TensorShape b);Asserts that two dimension sizes are equal.
void cten_assert_dim(const char* title, int a, int b);Internal function to perform broadcasting on two tensors for element-wise operations.
bool cten_elemwise_broadcast(Tensor* a, Tensor* b);