Programming libraries for audio, graphics and media development

.

Here are listed some libraries and APIs, that may be helpful when developing audio, video and media applications.

Note: This Page should include only libraries (APIs) that are cross-platform and have an OSI or FSF (FSFE) accepted open-source license.

.

.

Jack audio API - http://jackaudio.org/ - cross platform audio/midi IO

Portmedia - http://portmedia.sourceforge.net/ with its sub-projects - port audio and port midi

Pulseaudio - http://pulseaudio.org/ .

OpenML - http://www.khronos.org/openml/

RtMidi - http://www.music.mcgill.ca/~gary/rtmidi/ - Cpp library for midi IO

WebAudio - http://www.w3.org/TR/webaudio/ - W3C WebAudio API 2012

• UGen++ audio processing API synthesis framework in C. It provides oscillators, filters, envelopes and event management etc. * [[http://code.google.com/p/pyo/ Python digital signal processing module ]] written in C to help digital signal processing script creation. * Jamoma DSP - interface for C, with an emphasis on real-time signal processing

* gtkIOStream jackd client - trivialises using jackd for audio input and output DSP processing.

• real-time audio programming 101

'Atomic Operations' are not allowed to be interrupted (e.g. by other threads).

• Atomics in glib
• OpenPA - Portable Atomics
• gcc - Atomic Builtins
• HP atomic_ops project
• Boost Atomic (C++ or C++11 or C++0x)
• Boost Atomic.hpp (c++)
• Intel® Threading Building Blocks (Intel TBB)
• Userspace RCU library - User-level Read-Copy Update, Lock-free data structures (RCU lock-free hash table with auto-resize, queues, stacks), atomic operations, efficient user-level wait/wake-up
• libsync - Atomic operations and userspace RCU
• PJLIB - A small footprint framework library written in C for making scalable applications. Includes Atomic Ops.
• gtkIOStream thread and gtkIOStream mutex - Set of mutex and thread Cheaders to trivialise threading. Can use either pthreads or GLib threads. \\ \\ ===== Lockfree (non-blocking) Data Structures - Libraries ===== \\ * [[http://lttng.org/urcu Userspace RCU (read-copy-update) ]] * What is RCU - Article * FastFlow - Lockfree FIFO Queues * Atomic Ptr Plus * nbds - non-blocking data structures * libLFDS * lock-free library of the University Cambridge * Rig library - lock-free data structures, safe strings and misc others C library * ARP - Apache Portable Runtime * RefCount library - lock-free word-based atomic reference counting * Sunrise Data Dictionary for C - Library for lock-free hash table storage of arbitrary data * Boost.Lockfree - A boost-like, but NOT boost, library for lockfree structs. * Noble library - a library of non-blocking synchronization protocols * Userspace RCU library liburcu-cds - Userspace RCU Lock-free data structures (RCU lock-free hash table with auto-resize, queues, stacks) * varchunk - rt-safe, lock-free, zero-copy, contiguous-chunk ringbuffer optimized for event handling.
  ===== Parallel Programming =====
  * Par4all compiler - Generates OpenCL, CUDA or OpenMP parallel code from (sequenciel) C code.
  * OpenMP (Open Multi-Processing) - OpenMP is an API (Application Programming Interface) that supports multi-platform shared memory multiprocessing programming. * Gomp - GNU implementation of the OpenMP spec. * (WP) OpenMP
  * Open-MPI (message passing interface) - MPI addresses primarily a message-passing parallel programming model. * MPI 2.2 specification * MPICH2 - open MPI implementation * mpi 4 py - MPI for Python - mpi 4 py Documentation * (WP) MPI - Message Passing Interface * MPI FAQ
  * MPI-RT 1.1 spec. - real-time message-passing-interface v1.1 (an implementation doesn't exist !?)
  * OpenCL - Standard for cross-platform, parallel programming of modern processors. * OpenCL develop-kit Linux - OpenCL defines an extended C language for writing compute kernels and a set of APIs for creating and managing these kernels. * OpenCL API * OpenCL Tutorial * PyOpenCL - use OpenCL parallel computation API from Python * PyFFT - fast fourier transform with OpenCL (or Cuda) using Python
  
  * Unified Parallel C - UPC is an extension of the C programming language designed for high-performance computing. * GCC-UPC - GNU implementation of the UPC language spec.
  * CSP For Posix Threads - “«Communicating Sequential Processes» is an algebra for analysing the behaviour of concurrent systems … a library … that makes programming with threads much easier.”
  
  * Parallel Computing Manual * Parallel Programming - Basic Theory
  
  see also: * The concurrency kit - An online Manual * (WP) IPC Methods (Inter-Process Communication) * http://www.cs.kent.ac.uk/projects/ofa/c%2B%2Bcsp/ - CppCSP (Communicating Sequential Processes) - Easy Concurrency for Cpp * (WP) concurrent programming languages 1 * (WP) Concurrent programming languages 2 * (WP) CSP - Communicating sequential processes * | RTLinux IPC * Posix Threads * IPC Mechanisms (Inter-Process-Communication)
  ===== Session handling =====
  Restore application-state and audio/midi connections.
  There are at least three different APIs for session handling in linux. They allow to save and restore the state of different applications, including midi and audio connections !
  * JackSession - is kind of basic, requires applications to be restarted * Ladish - is advanced, has a concept of storing session-“rooms” * NSM - has most of the really useful functions (based on OSC) - does not require apps to be restarted
  For more information, visit the projects websites: * NSM - The NON Session-Manager - comes as part of NON-DAW - API Link and screenshot * JackAudio Session Handler * ladish - Linux Audio Dev. Session Handler
  
  ===== Thread management and CPU-affinity in linux ===== * there is a tool called schedtool, which allows to “set all CPU scheduling parameters Linux is capable of or display information for given processes.” * also to mention: nice and renice tools (part of the “bsdutils” package) * and linux API functions: sched_setscheduler(), sched_setaffinity()
  Related MAN-Pages: * schedtool (formerly: schedutils) * perf-sched - Trace or measure scheduler properties (latencies) (perf in package “linux-tools-common”) * sched_setaffinity * sched_setscheduler * nice - note: don't confuse nice priorities with real-time-priorities (in package “bsdutils”) * chrt (part of "util-linux" package - a tool to maniplate real-time attributes (priorities)
  Related Articles: * example * Take charge of processor affinity * The Linux-Kernel API (2.6.20) * real-time kernel wiki - more tools and information * About Linux Processes * task_struct * CPU Affinity - LinuxJournal * sched.h - header file * GNU C - CPU Priority And Scheduling * Linux Driver HowTO (german!) * Setting processor affinity * list running processes * kernel hacking
  * hwloc - portable multi threading
  Part of an Answer at stackoverflow.com :

  sched_setaffinity() simply tells the scheduler which 
  CPUs is that process/thread allowed to run on, then calls 
  for a re-schedule.
  
  The scheduler actually runs on each one of the CPUs, so it 
  gets a chance to decide what task to execute next on 
  that particular CPU.
  
  If you're interested in how you can actually call some code 
  on other CPUs, I suggest you take a look at 
  smp_call_function_single(). In case we want to call something 
  on another CPU, this calls generic_exec_single(). 
  The latter simply adds the function to the target CPU's 
  call queue and forces a reschedule through some 
  IPI stuff (if the queue was empty).
  
  Bottom line is: there no actual SMP variant of 
  the _jmp_ instruction. Instead, code running on other CPUs 
  cooperates in order to accomplish the task.
  

  
  
  
  ===== GUI libraries - create graphical user interfaces ===== Comment: additional info would be nice, for example: does it support skins/themes, download size, is it for graphics only or a whole framework… ==== 2-dimensional ==== Juce (framework) - Juce and Juced Sfml - http://www.sfml-dev.org/features.php . Lgi - http://www.memecode.com/lgi.php OpenFrameworks (framework) - http://www.openframeworks.cc/about . Fox toolkit - http://www.fox-toolkit.org/ . Qt (framework) - http://qt.nokia.com/products . Gtk - http://www.gtk.org/ . Fltk - http://www.fltk.org/ . WxWidgets - http://www.wxwidgets.org/ . Ultimate++ - http://ultimatepp.org/www$uppweb$overview$en-us.html - Cpp cross-platform Framework . gtkIOStream - http://sourceforge.net/projects/gtkiostream/ - Inline headers for C iostream style GTK GUI building .

LibNui - http://www.libnui.net/pages/about.php .

Coin3d - http://www.coin3d.org/lib/about_coin3d/coin3d-overview .

Clutter - http://www.clutter-project.org/about .

COGL - http://www.clutter-project.org/documentation (is a part of Clutter).

Visualization Library - http://www.visualizationlibrary.com/jetcms/

VTK - http://www.vtk.org/

OpenInventor - http://oss.sgi.com/projects/inventor/ .

Agar - http://wiki.libagar.org/wiki/Main_Page - http://libagar.org/index.html.en .

SDL - http://www.libsdl.org/ .

glfw - http://www.glfw.org/

guichan - http://guichan.sourceforge.net/wiki/index.php/About

glam - http://glam.sourceforge.net/ (rel.2004-02)

gigi - http://gigi.sourceforge.net/ (rel.2008-03)

GLV gui - http://mat.ucsb.edu/glv/

MyGUI - http://www.ogre3d.org/addonforums/viewtopic.php?f=17&t=7488 (rel.2010-02) uses OGRE game-lib

MiniGUI - http://www.minigui.org/res.shtml for embedded devices

ClanLib - http://clanlib.org/wiki/Main_Page game-lib with gui-toolkit

CEGUI - http://www.cegui.org.uk/wiki/index.php/Main_Page

Allegro - http://alleg.sourceforge.net/readme.html

.

Disko - http://www.diskohq.com/ (embedded library with OpenGL support)

MeeGo - http://meego.com/about (for embedded)

Moblin - http://moblin.org/about-moblin Moblin joined MeeGo

MoSync - http://www.mosync.com/ - Cpp, for Smartphones etc.

Kivy - http://kivy.org/ - Python library for rapid development, innovative ui, multi-touch apps

Tizen - https://www.tizen.org/ - “Environment for application developers, based on HTML5. ”

.

AGG Anti Grain Geometry - http://www.antigrain.com/

GGI - http://www.ggi-project.org/ - graphic-system abstraction layer

Cairo Graphics - http://cairographics.org/

Batik - http://xmlgraphics.apache.org/batik/

LibSugar - http://libsugar.sourceforge.net/

librsvg - http://librsvg.sourceforge.net/

.

OpenGL Mathematics library (GLM) - http://glm.g-truc.net/about.html

CML (Configurable Math Library) - http://cmldev.net/?page_id=7

TLSF - http://rtportal.upv.es/rtmalloc/ - for real-time memory allocation

.

.