TarsosDSP


Source link: https://github.com/JorenSix/TarsosDSP

TarsosDSP

TarsosDSP is a Java library for audio processing. Its aim is to provide an easy-to-use interface to practical music processing algorithms implemented, as simply as possible, in pure Java and without any other external dependencies. The library tries to hit the sweet spot between being capable enough to get real tasks done but compact and simple enough to serve as a demonstration on how DSP algorithms works. TarsosDSP features an implementation of a percussion onset detector and a number of pitch detection algorithms: YIN, the Mcleod Pitch method and a “Dynamic Wavelet Algorithm Pitch Tracking” algorithm. Also included is a Goertzel DTMF decoding algorithm, a time stretch algorithm (WSOLA), resampling, filters, simple synthesis, some audio effects, and a pitch shifting algorithm.

To show the capabilities of the library, TarsosDSP example applications are available. Head over to the TarosDSP release directory for freshly baked binaries and code smell free (that is the goal anyway), oven-fresh sources.

Some information about TarsosDSP can be found in the paper TarsosDSP, a Real-Time Audio Processing Framework in Java, by Joren Six, Olmo Cornelis, and Marc Leman, in Proceedings of the 53rd AES Conference (AES 53rd), 2014. If you use TarsosDSP in academic research, please cite this paper.

@inproceedings{
six2014tarsosdsp,
author

= {
Joren Six and Olmo Cornelis and Marc Leman
}
,
title

 = {
{
TarsosDSP, a Real-Time Audio Processing Framework in Java
}

}
,
booktitle
= {
{
Proceedings of the 53rd AES Conference (AES 53rd)
}

}
,
 year

  =  2014 
}

Quickly Getting Started with TarsosDSP

Head over to the TarsosDSP release repository and download the latest TarsosDSP library. Include the Jar-file in your project and you are ready to go. To get up to speed quickly, check the TarsosDSP Example applications for inspiration and consult the API documentation.

Every release of TarsosDSP contains the following:

  • TarsosDSP-x.x-Documentation/ JavaDoc documentation
  • TarsosDSP-x.x-Examples/ Example applications.
  • TarsosDSP-Android-x.x-bin.jar Jar library for inclusion in Android projects, withouth source files included.
  • TarsosDSP-Android-x.x.jar Jar library for inclusion in Android projects, with source files included.
  • TarsosDSP-x.x-Manual.pdf A manual describing the core concepts of TarsosDSP.
  • TarsosDSP-x.x-Readme.html This readme.
  • TarsosDSP-x.x-bin.jar Jar library for inclusion in Java projects, withouth source files included.
  • TarsosDSP-x.x.jar Jar library for inclusion in Java projects, with source files included.

The source code of TarsosDSP are compatible with Java 1.6. The jar files are compiled for 1.7.

TarsosDSP on Android

If you want to do audio processing on Android TarsosDSP is a great fit. The main distribution has no dependencies on javax.sound.xxx and does work well on Android by default. To add TarsosDSP to an Android Studio project download the latest TarsosDSP Android release and place it in the /app/libs/ folder of your project. By default Gradle includes jar files placed in that directory in your classpath.

To check if this is the case look for the following lines in the build.gradle file in the module app:

dependencies {

  compile fileTree(dir: 'libs', include: ['*.jar']) 
}
 

For futher information, please consult TarsosDSP on Android – Audio Processing in Java on Android.

TarsosDSP Example Applications

TarsosDSP contains some ready made example applications. Most have a Java Swing user interface. They show which functionality is present in the library and how to use it.

  • SoundDetector show how you loudness calculations can be done. When input sound is over a defined limit an event is fired.
  • PitchDetector this demo application shows real-time pitch detection. When pitch is detected the hertz value is printed together with a probability.
  • PercussionDetector show the percussion (onset) dectection. Clapping your hands causes an event. This demo application also shows the influence of the two parameters on the algorithm.
  • UtterAsterisk a game with the goal to sing as close to a melody a possible. Technically it shows real-time pitch detection with YIN or MPM.
  • Spectrogram in Java shows a spectrogram and detected pitch, either live or from an audio file. It is interesting to see which frequencies are picked as fundamentals.
  • Goertzel DTMF decoding an implementation of the Goertzel Algorithm. A fancy user interface shows what goes on under the hood.
  • Audio Time Stretching – Implementation in Pure Java Using WSOLA an implementation of a time stretching algorithm. WSOLA makes it possible to change the play back speed of audio without changing the pitch. The play back speed can be changed at any moment, even when there is audio playing.
  • Audio Feature Extraction a command line application to do simple feature extraction.
  • Audio Synthesis a command line application to do simple audio synthesis.
  • Pitch Shifting an example application that does pitch shifting, either in real-time on a microphone input, or on recorded audio. Also included is a command line application to do pitch shifting.

Developing TarsosDSP

If you want to build from source, or want to improve TarsosDSP follow the instructions below. Contributions to TarsosDSP are more than welcome, if you have a an algorithm to add or find a bug, do not hesitate to send me a message.

TarsosDSP uses Apache Ant as a build system. The instructions below detail how you can build from source. When everything runs correctly you should be able to run all example applications and have the latest version of the TarsosDSP library for inclusion in your projects. Also the Javadoc documentation for the API should be available.

TarsosDSP with Ant

To you need Apache Ant and git installed on your system. The following commands fetch the source and build the library and example jars:

git clone https://[email protected]/JorenSix/TarsosDSP.git cd TarsosDSP cd build ant tarsos_dsp_library #Builds the core TarsosDSP library ant build_examples #Builds all the TarsosDSP examples ant javadoc #Creates the documentation in TarsosDSP/doc 

Source Code Organization & Developing

The library is separated into five source folders: 1) the main core functionality in src/core, TarsosDSP example applications in src/examples, unit tests in src/test, JVM audio I/O in src/jvm and Android audio I/O in src/android.

  • src contains the source files of the DSP library.
    • src/core contains the main core classes.
    • src/test contains unit tests for some of the DSP functionality.
    • src/examples contains a couple of example applications with a Java Swing user interface.
    • src/android contains the source files for audio I/O on Android. It is dependent on the Android Runtime.
    • src/jvm contains the source files for audio I/O on JRE. It is dependent on the Java Runtime Environment.
    • src/patcher I/O for patcher environments like pure data and Max/MSP. It shows how TarsosDSP and pd, MaxMSP can connect.
  • build contains ANT build files. Either to build Java documentation or runnable JAR-files for the example applications.
  • lib although the TarsosDSP core does not require any external dependencies the lib folder does contain two jar-file to easily run unit-tests (JUnit 4 and Hamcrest). It also contains a pure data library when one wants to develop for the pure data environment.

To make development with Eclipse easy, make sure the subfolders of src are marked as “source folder”, and not the src folder itself (as is usually the case). For Android development exclude the jvm folder and include android and link to an Android runtime of your choosing.

Credits

TarsosDSP was developed at University College Ghent, School of Arts between 2009 and 2013, from late 2013 the project is supported by University Ghent, IPEM.

The TarsosDSP borrows algorithms from various other libraries or research paper. Below a complete list of credits can be found.

Changelog






Version 1.0
2012-04-24
First release which includes several pitch trackers and a time stretching algorithm, amongst other things. Downloads and javadoc API can be found at the TarsosDSP release directory
Version 1.1
2012-06-4
Changed how the audio dispatcher stops. Added StopAudioProcessor. Added FastYin implementation by Matthias Mauch Added AMDF pitch estimator by Eder Souza
Version 1.2
2012-08-21
Modified the interface of PitchDetector to return a more elaborate result structure with pitch, probability and a boolean “is pitched”. Added an implementation of an envelope follower or envelope detector.
Version 1.3
2012-09-19
TarsosDSP can do audio synthesis now. The first simple unit generators are included in the library. It has a new audio feature extraction feature, implemented in the FeatureExtractor example. Added ASCII-art to the source code (this is the main TarsosDSP 1.3 feature).
Version 1.4
2012-10-31
Included a resample feature, implemented by libresample4j. Together with the WSOLA implementation, it can be used for pitch shifting (similar to Phase Vocoding). A pitch shifting example (both with a CLI and a UI) is added in the 1.4 version of the TarsosDSP library as well.
Version 1.5
2013-04-30
Converted TarsosDSP to maven. This is known as the Malaryta-release. The “Malaryta” release is provided to you by RikkiMongoose (idea, documents, git things) and Ultar (converting to maven, refactoring). Malaryta is the capital of Malaryta Raion, Brest Region in the Republic of Belarus. Both of developers spent their childhood in Brest, and think that title Malaryta is as strange as Ubuntu or Whistler. The 1.5 release also includes various FFT window functions from the cool Minim project by Damien Di Fede.
Version 1.6
2013-06-12
This release features practical onset and beat detection algorithms. A complex domain onset detection and a spectral flux onset detection algorithm are added. This release also includes a way to guess a beat from onsets. Parts of the BeatRoot system , by Simon Dixon, are included to this end. Also included in this release is an implementation of the Constant-Q transform.
Version 1.7
2013-10-08
This release adds the ability to extract the MFCC from an audio signal. Also an example of the Constant-Q transform is added, together with a reusable visualization class library. The build system is reverted back to pure ANT
Version 1.8
2014-04-10
With this release it is possible to extract spectral peaks from an FFT and get precise frequency estimates using phase info. An example application called SpectralPeaks is added as well.
Version 1.9
2014-08-10
This release includes a Haar Wavelet Transform and an example of an audio compression algorithm based on Haar Wavelets. It also includes a significant change in package naming.
Version 2.0
2014-08-13
The 2.0 version is worth the major version update since it offers out-of-the-box support for Android. The release has no more dependencies on the parts of the java runtime that are not included in Android. To offer this support some packages have been shifted around. The code that does I/O is dependent on the runtime (JVM or Dalvik) and is abstracted using the be.tarsos.dsp.io package.
Version 2.1
2015-03-03
The 2.1 version restructures some of the source files. All source can now be found in src. The ant build file is adapted to reflect this change. This version also includes an STFT pitch shifter. There was already a time domain pitch shifter included and now a frequency domain implementation is present as well.
Version 2.2
2015-03-03
The 2.2 version includes a new AudioDispatcher . It has been reviewed toroughly and now behaves predictably for the first and last buffers as well. To prevent compatibility issues the version has been changed.
Version 2.3
2015-09-01
The 2.3 version includes improved Android support: audio decoding on Android can be done using a provided, statically compiled ffmpeg binary. The ffmpeg decoding functionality for JVM has been improved as well. If no ffmpeg executable is found it is downloaded automatically from here .
Version 2.4
2016-12-01
Some small changes to the pipe decoder. Now it is possible to set a start and duration for the incoming decoded audio from ffmpeg.

Resources

A simple and custom calendar view.

AnyMaps allows you to use the same API for different maps providers without the need to adjust existing Google Maps implementation (apart from changing the package name).

A simple ViewPager with parallax effect.

shapeletter is a simple view that displays a letter within circle or rectangle much like the gmail app.

ExVidPlayer is simple library that abstracts ExoPlayer.

Features:

  • Gesture controls.
  • Brightness controls with one single touch vertical scroll.
  • Volume controls with multitouch vertical scroll.
  • Forward on left swipe.
  • Reverse on right swipe.
  • Player controls

Data binding library for Android which is:

  • Lightweight
  • Doing one thing and doing it well
  • Easily extendable
  • Fully covered with unit tests
  • Working great with Jack, Retrolambda or Kotlin

Topics


2D Engines   3D Engines   9-Patch   Action Bars   Activities   ADB   Advertisements   Analytics   Animations   ANR   AOP   API   APK   APT   Architecture   Audio   Autocomplete   Background Processing   Backward Compatibility   Badges   Bar Codes   Benchmarking   Bitmaps   Bluetooth   Blur Effects   Bread Crumbs   BRMS   Browser Extensions   Build Systems   Bundles   Buttons   Caching   Camera   Canvas   Cards   Carousels   Changelog   Checkboxes   Cloud Storages   Color Analysis   Color Pickers   Colors   Comet/Push   Compass Sensors   Conferences   Content Providers   Continuous Integration   Crash Reports   Credit Cards   Credits   CSV   Curl/Flip   Data Binding   Data Generators   Data Structures   Database   Database Browsers   Date &   Debugging   Decompilers   Deep Links   Dependency Injections   Design   Design Patterns   Dex   Dialogs   Distributed Computing   Distribution Platforms   Download Managers   Drawables   Emoji   Emulators   EPUB   Equalizers &   Event Buses   Exception Handling   Face Recognition   Feedback &   File System   File/Directory   Fingerprint   Floating Action   Fonts   Forms   Fragments   FRP   FSM   Functional Programming   Gamepads   Games   Geocaching   Gestures   GIF   Glow Pad   Gradle Plugins   Graphics   Grid Views   Highlighting   HTML   HTTP Mocking   Icons   IDE   IDE Plugins   Image Croppers   Image Loaders   Image Pickers   Image Processing   Image Views   Instrumentation   Intents   Job Schedulers   JSON   Keyboard   Kotlin   Layouts   Library Demos   List View   List Views   Localization   Location   Lock Patterns   Logcat   Logging   Mails   Maps   Markdown   Mathematics   Maven Plugins   MBaaS   Media   Menus   Messaging   MIME   Mobile Web   Native Image   Navigation   NDK   Networking   NFC   NoSQL   Number Pickers   OAuth   Object Mocking   OCR Engines   OpenGL   ORM   Other Pickers   Parallax List   Parcelables   Particle Systems   Password Inputs   PDF   Permissions   Physics Engines   Platforms   Plugin Frameworks   Preferences   Progress Indicators   ProGuard   Properties   Protocol Buffer   Pull To   Purchases   Push/Pull   QR Codes   Quick Return   Radio Buttons   Range Bars   Ratings   Recycler Views   Resources   REST   Ripple Effects   RSS   Screenshots   Scripting   Scroll Views   SDK   Search Inputs   Security   Sensors   Services   Showcase Views   Signatures   Sliding Panels   Snackbars   SOAP   Social Networks   Spannable   Spinners   Splash Screens   SSH   Static Analysis   Status Bars   Styling   SVG   System   Tags   Task Managers   TDD &   Template Engines   Testing   Testing Tools   Text Formatting   Text Views   Text Watchers   Text-to   Toasts   Toolkits For   Tools   Tooltips   Trainings   TV   Twitter   Updaters   USB   User Stories   Utils   Validation   Video   View Adapters   View Pagers   Views   Watch Face   Wearable Data   Wearables   Weather   Web Tools   Web Views   WebRTC   WebSockets   Wheel Widgets   Wi-Fi   Widgets   Windows   Wizards   XML   XMPP   YAML   ZIP Codes