Overview of the scientific computing ecosystem

This chapter aims to provide a rough overview of the ecosystem for scientific computing with the aim to present the available packages with an overview of what they can be used for.

In general, keep the Nimble directory handy to search for Nim packages from your browser.

Note that due to the way this page is laid out, some packages might appear multiple times under different sections.

Further: if you feel any existing library is missing on this page, please either create a PR to this page, open an issue or simply write a message in the Matrix/Discord Nim science channel!

Fundamental data handling libraries

The libraries listed here all provide basic data types that are helpful in general (multidimensional arrays & data frames).

  • Arraymancer ⇐ provides a generic Tensor[T] type, similar to a Numpy ndarray. On top it defines operations from indexing, broadcasting and apply/map/fold/reduce operations to linear algebra and much more
  • Neo ⇐ provides primitives for linear algebra. This means it implements vectors and matrices, either with static or dynamic sizes.
  • Datamancer ⇐ builds on top of Arraymancer to provide a runtime based DataFrame implementation. Runtime based means the types of columns are determined at runtime instead of compile time (e.g. via a schema). The focus is on column based operations.
  • NimData ⇐ provides another DataFrame implementation, which - compared to Datamancer - has a stricter CT safety focus. Its implementation is row based and the DataFrame type is determined at compile time. Operations are built on top of iterators for lazy evaluation.

Data visualization

There are multiple libraries for data visualization ("plotting") available, each with their own focus and thus pros and cons.

Beyond the libraries listed in this section, keep in mind that your favorite Python, Julia and R plotting library is only a nimpy, nimjl and Rnim call away!

Numerical algorithms

Numerical algorithms for integration, interpolation, (numerical) differentiation and solving differential equations are of course fundamental for scientific computing.

  • Numericalnim ⇐ the most comprehensive library for numerical algorithms in Nim. Also see the integration tutorial here.
  • Polynumeric ⇐ provides common operations (derivatives, root finding, etc.) of polynomials

Optimization

Optimization (possibly non-linear) problems are a problem domain large enough to deserve their own section beyond the "numerical algorithm" section.

  • Numericalnim ⇐ provides some algorithms for non-linear optimization. These include Levenberg-Marquardt for non-linear curve fitting and (L)BFGS for general optimization problems.
  • fitl ⇐ contains a pure Nim linear least squares solver (so no LAPACK dependency!) and provides many goodness-of-fit tests
  • nimnlopt ⇐ wrapper of the NLopt C library. It includes a large number of algorithms for non-linear optimization problems (gradient / non gradient & local / global methods) with support for constraints.
  • nim-mpfit ⇐ wrapper of the C library cmpfit, an implementation of the Levenberg-Marquardt algorithm for non-linear least squares problems (i.e. non-linear curve fitting).
  • gsl-nim ⇐ wrapper for the GNU Scientific Library, which probably

satisfies all your numerical optimization needs (and much more), if you can live with the GSL dependence and raw C API.

(Binary) data storage & serialization libraries

  • nimhfd5 ⇐ high level bindings for the HDF5 library
  • netcdf ⇐ wrapper for NetCDF library
  • mcpl ⇐ wrapper for MCPL library
  • Arrow
  • nio ⇐ also includes operations for binary data handling
  • nimcfitsio ⇐ wrapper for the CFITSIO library, typically used in astronomy
  • nim-teafiles ⇐ library to read TeaFiles, a format for fast read/write access to time series data
  • CSVtools ⇐ library for typed iterators on CSV files
  • DuckDB ⇐ DuckDB wrapper for Nim. DuckDB is a DB focused on fast data analysis

Linear algebra

A list of libraries for linear algebra operations. These libraries typically provide their own matrix and vector types and define common (and not so common) operations on them.

  • Neo ⇐ linear algebra library with support for dense and sparse matrices. Wraps BLAS & LAPACK and also has GPU support
  • Manu ⇐ pure Nim library for operations on real, dense matrices (solving linear equations, determinants, matrix inverses & decompositions, ...)
  • Arraymancer ⇐ Arraymancer also provides many linear algebra routines
  • gsl-nim ⇐ GSL provides many linear algebra routines

Algebra

  • emmy ⇐ Algebraic structures and operations on them
  • nim-algebra ⇐ implements many routines for rings, fields and groups

Symbolic operations

Libraries dealing with symbolic instead of numeric operations.

  • astgrad ⇐ symbolic derivatives based on Nim AST
  • symbolicnim ⇐ pure Nim library for symbolic computations
  • symengine ⇐ wrapper for C++ library for symbolic computations

Number types

These libraries all provide specific data types suited to certain kind of operations.

  • decimal libraries
    • nim-decimal ⇐ decimal library wrapping C lib mpdecimal
    • decimal128 ⇐ pure Nim decimal library, missing some features
  • multi-precision integers (bigints)
    • bignum ⇐ wrapper of GMP providing arbitrary precision ints & rationals, does not wrap mpfr (so no multi precision floats)
    • bigints ⇐ pure Nim bigint library
    • theo ⇐ optimized bigint library, WIP
  • fpn ⇐ fixed point number library in pure Nim
  • stdlib rationals ⇐ Nim standard library module for rational numbers
  • stdlib complex ⇐ Nim standard library module for complex numbers

Statistics, sampling and random number generation

  • statistical-tests
  • linear-models
  • distributions
  • stdlib stats ⇐ basic statistics module from the stdlib. Supports moments up to kurtosis & provides basic regression support
  • alea ⇐ library for sampling from many different distribiutons. Allows to wrap custom (e.g. stdlib) RNGs
  • sitmo ⇐ Nim implementation of the Sitmo parallel RNG
  • stdlib random ⇐ random number generation of the Nim standard library
  • nim-random ⇐ alternative to the Nim stdlib random number library
  • nim-mentat ⇐ implements exponentially weighted moving averages
  • fitl ⇐ contains a submodule dists for random sampling, mixing of distributions, (inverse) CDFs

Machine learning

  • Flambeau ⇐ as a wrapper to libtorch provides access to state-of-the-art ML features
  • Arraymancer ⇐ Arraymancer implements a DSL to define neural networks (see the examples) and provides other, more primitive ML tools (PCA, ...)
  • exprgrad ⇐ Experimental deep learning framework, based on an easily extensible LLVM compiled differentiable programming language
  • DecisionTreeNim ⇐ implements decision trees & random forests

Natural language processing

  • tome ⇐ provides tokenization and parts of speech (POS) tagging
  • word2vec ⇐ Word2vec implementation in Nim
  • fastText ⇐ library to perform predictions of fastText models
  • scim ⇐ library for helpful tools for speech recognition based on arraymancer

Spatial data structures, distance measures & clustering algorithms

  • kdtree ⇐ k-d tree implementation in pure Nim
  • RTree ⇐ R- and R*-Tree implementations in pure Nim
  • QuadtreeNim ⇐ Quadtree implementation implementation in pure Nim
  • distances ⇐ library to compute distances under different metrics with support for standard sequences, arraymancer & neo types
  • arraymancer ⇐ arraymancer contains a k-d tree implementation, multiple distance metrics (incl. user defined custom metrics) plus k-means & DBSCAN clustering algorithms
  • spacy ⇐ collection of different spatial data structures
  • DelaunayNim ⇐ library to compute the Delaunay triangulation of a set of points
  • nim-mentat ⇐ implements Balanced Box-Decomposition trees

Special functions

These libraries implement different special functions.

  • stdlib math ⇐ The Nim standard library math module contains all libraries you find in math.h.
  • spfun ⇐ library for many special functions used in stats, physics, ...
  • gsl-nim ⇐ The GSL probably provides any special function you may need
  • special-functions ⇐ contains many special functions, which are not part of the stdlib module

FFT

  • nimfftw3 ⇐ FFTW3 wrapper
  • impulse ⇐ pocket FFT wrapper, in principle a repository for signal processing primitives
  • kissFFT ⇐ kissFFT wrapper

Primitive compute wrappers

Multithreading, multiprocessing & asynchronous processing

  • weave ⇐ very low overhead, high performance multithreading runtime
  • taskpools ⇐ lightweight threadpool implementation
  • threadpools ⇐ Custom threadpool implementation
  • threading ⇐ New pieces for multithreading in times of ARC/ORC
  • asynctools ⇐ Various async tools for usage with Nim's stdlib async macro
  • asyncthreadpool ⇐ An awaitable threadpool implementation
  • cligen ⇐ contains a procpool submodule for easy multiprocessing

Biology

  • hts-nim ⇐ A wrapper for htslib for Nim for parsing of genomics data files
  • bionim ⇐ collection of data structures and algorithms for bioinformatics
  • bio ⇐ a library for working with biological sequences

Physics & astronomy

  • unchained ⇐ library for CT checking of physical units and automatic conversion between units
  • qex ⇐ lattice QCD library
  • mclimit ⇐ Nim port of the ROOT TLimit class for confidence level computations (limits) for experiments with small statistics
  • nim-constants ⇐ contains many physical and mathematical constants
  • astroNimy ⇐ astronomical image processing library
  • orbits ⇐ library for orbital mechanics calculations
  • nim-root ⇐ partial wrapper for CERN's ROOT
  • MDevolve ⇐ integrator framework for molecular dynamic evolutions
  • polypbren ⇐ program to compute renormalized parameters of charged colloids
  • xrayAttenuation ⇐ for calculations of X-ray transmissions through materials & X-ray reflectivity under grazing angles

Mathematics

  • perms-nim ⇐ library for permutation group calculations and factorization algorithms

Other useful libraries

  • scinim ⇐ library of general scientific things that are either primitives or too small to have their own library
  • Measuremancer ⇐ library for automatic error propagation of measurement uncertainties
  • gsl-nim ⇐ wrapper for GSL (GNU Scientific Library)
  • nim-opencv ⇐ Nim wrapper for OpenCV
  • zero-functional ⇐ library for zero cost chaining of functional primitves (map, apply, fold, ...). Fuses multiple operations into a single loop.
  • iterrr ⇐ another library for zero cost chaining, similar to zero-functional. Aims to be easier to extend.
  • flower ⇐ pure Nim bloom filter, probabilistic data structure to check if elements are in a set ("possibly in set" vs. "definitely not in set"). Supports arbitrary Nim types in single filter.
  • pattern matching:
  • Synthesis ⇐ DSL to generate statically checked state machines
  • jupyternim ⇐ Jupyter kernel for Nim
  • cligen ⇐ elegant library to write CLI interfaces
  • LatexDSL ⇐ DSL to generate CT checked latex strings, supporting Nim variable interpolation
  • nim-mathexpr ⇐ mathematical string expression evaluator library
  • nim-pari ⇐ wrapper for the PARI C library underlying the PARI/GP computer algebra system
  • memo ⇐ macro library to allow memoization of function calls (automatic caching of function calls)
  • forematics ⇐ Nim implementation of a Metamath verifier
  • DeepLearningNim ⇐ example of building a DQN with arraymancer

Educational resources

  • nim-bayes ⇐ Tutorial about Bayesien Inference of a linear model in Nim

Language bindings

First of all Nim itself of course provides direct support to wrap C and C++ libraries using its FFI. See the Nim manual here for an introduction to the C / C++ FFI.

For more details on how to use the language specific bindings, see the section about it here

Direct language bridges

  • nimjl ⇐ bridge to Julia
  • nimpy ⇐ bridge to Python
  • Rnim ⇐ bridge to R

Tools to wrap C / C++

  • c2nim ⇐ the default Nim tool to generate Nim wrappers of C header files
  • futhark ⇐ automatic imports of C header files in Nim code
  • nimterop ⇐ library to simplify wrapping of C/C++ using tree-sitter