The DSP Audio Toolkit for Flash. The DATF is a wrapper class that provides simple interfacing with C/C++ DSP functions via Adobe Alchemy.
| DATF.as | The DSP Audio Toolkit for Flash. |
| Usage | The DATF is designed for developers who want more control over the dynamic audio capabilities of Actionscript. |
| Example | |
| Constructor | |
| DATF | Constructor to create a DATF object. |
| Audio Framing | |
| setFrame | This function copies the data from a ByteArray into the shared Actionscript/C memory. |
| Basic Library Functions | |
| FFT | Computes the Fast Fourier Transform (FFT) of the current frame. |
| IFFT | Performs an Inverse Fourier Transform on the current frame. |
| magSpectrum | Calculates the magnitude spectrum from the complex frequency spectrum. |
| Spectral Features | All spectral features are computed on mono audio. |
| getBandwidth | Calculates the spectral bandwidth for the current frame. |
| getCentroid | Calculates the spectral centroid for the current spectral data. |
| getFlux | Calculates the change in spectral energy between the current frame and the previous frame. |
| getIntensity | Calculates the spectral intensity for the current frame. |
| getRolloff | Calculates the spectral rolloff for the current frame. |
| Spectral Analysis | |
| getHarmonicAmplitudes | A function to return the amplitude of the harmonics. |
| getHarmonicFrequencies | Returns an array containing the partial’s frequencies populated by callilng the getHarmonics function. |
| getHarmonics | Isolates the harmonics (or partials more generally speaking) of spectral data. |
| getLPC | Calculates the linear prediction coefficients using Levinson-Durbin recursion. |
| getPitch | Uses the autocorrelation method to estimate the pitch of frame. |
| getPitchFromSpeech | Same as getPitch except it provides min/max pitch values and a threshold useful for determining if there is an active signal in the current frame. |
| addReverb | This function implements a well-known image model in order to simulate room reverb based on simulated room dimensions. |
| filterAudioFIR | Filters the audio with the given filter (array of coefficients). |
| filterAudioIIR | Filters the audio with the given filter (array of coefficients). |
| vocoder | Applies the phase vocoder functionality to each AudioChannel object unique to the DATF |
| Utilities | |
| clearAudioBuffers | A Function to clean out the audioBuffers at the completion of audio file playback. |
| checkOutputBuffer | A function to determine if the C-based audioFrame is ready for reading during audio playback. |
| endOfFile() | This function should be called when a file is complete or after ALF.stopAudio has been called. |
| getChOutPtrs() | Returns: An array containing the pointers to the C audio buffers. |
| getCRAM() | Returns: A pointer to the C memory buffer. |
| getHopSize() | The current hopSize in the DATF. |
| getNumberOfChannels() | The current number of channels. |
| reInitializeChannel | This funciton should be called when a new song is loaded with a different sample rate(for wav) or the hop size is changed. |
| resetAll | This resets all of the buffers and values to their state at initialization. |
The DATF is designed for developers who want more control over the dynamic audio capabilities of Actionscript. For simpler ‘out of the box’ functionality, use the <ALF.as>. The DATF provides a means to access a variety of DSP functions that are written in C/C++ and then compiled with the Alchemy enabled g++ compiler to create bytecode that is optimized for the Actionscript Virtual Machine. The ALFPackage.swc file you include in your project (process outlined in the Introduction) contains all the functions that are called from the DATF. A simple example is given below. Note that this is a scaled down version of what ALF does. Again, the reason for using DATF rather than ALF is to implement the dynamic audio playback in a different manner.
class myClass{
var DSP:DATF;
var mp3Bytes:ByteArray:
var playBytes:ByteArray:
var audio:Sound;
var mp3Audio:Sounc;
var audioCh:SoundChannel;
var cRAM:ByteArray;
var audioPtrs:Array;
var leftBufferStatus:Array;
var rightBufferStatus:Array;
const var STEREO:Boolean = true;
const var hopSize:uint = 2048;
const var fs:uint = 44100;
public function myClass(){
var song:String = "Testsong.mp3";
var mp3Request:URLRequest = new URLRequest(song); // Load filename
mp3Bytes = new ByteArray(); // For raw samples from file
mp3Bytes.endian = Endian.LITTLE_ENDIAN;
audio = new Sound(); // Sound object for playback
mp3Audio = new Sound(); // Sound object for audio file sample extraction
audioCh = new SoundChannel();
cRAM = new ByteArray();
leftBufferStatus = new Array();
rightBufferStatus = new Array();
audioPtrs = new Array();
// Attempting to read in the MP3 file
try{
sourceAudio.load(mp3Request);
} catch(e:IOError){
trace('Error reading MP3 file');
}
mp3Audio.addEventListener(Event.COMPLETE, mp3Loaded);
audio.addEventListener(SampleDataEvent.SAMPLE_DATA, mp3AudioCallback); // Dispatched on each audio frame
}
//Load MP3 file and play
private function mp3Loaded(evt:Event):void{
//Initilize the DATF
DSP = new DATF(hopSize, STEREO, fs);
cRAM = DSP.getCRAM(); // Pointer to shared AS/C memory
audioPtrs = DSP.getChAudPtrs(); // Pointers to the L/R channels
// Read first frame
mp3Bytes.position = 0;
mp3Position += mp3Audio.extract(mp3Bytes,hopSize,mp3Position); // Extract samples
}
private function mp3AudioCallback(evt:SampleDataEvent):void {
mp3Bytes.position = 0;
DSP.loadMP3Frame(mp3Bytes); // Load the audio frame into the C buffer
// You must check to see if audio is being synthesized by DATF (i.e. using reverb)
leftBufferStatus = DSP.checkAudioBuffer("left");
rightBufferStatus = DSP.checkAudioBuffer("right");
numSamplesToPlay = Math.min(leftBufferStatus[1], rightBufferStatus[1]); // Find how many samples to play
for(i = 0; i < numSamplesToPlay; i++){
cRAM.position = audioPtrs[0] + i*sizeofFloat; //position for leftCh
leftSample = cRAM.readFloat();
cRAM.position = audioPtrs[1] + i*sizeofFloat; //position for rightCh
rightSample = cRAM.readFloat();
evt.data.writeFloat(leftSample);
evt.data.writeFloat(rightSample);
}
//Extract audio data from sound object
mp3Bytes.length = 0;
mp3Position += mp3Audio.extract(mp3Bytes,hopSize,mp3Position); // Extract samples
mp3Bytes.position = 0; // Reset pointer
//Tell C there is a new frame coming
DSP.newFrame();
}
}
public function DATF( _hopSize: uint, _STEREO: Boolean, _fs: uint, frameLookAhead: Number ):void
Constructor to create a DATF object. This constructor initializes the shared memory between Actionscript and C/C++. Only mono and stereo files are accepted. The C/C++ library is included via the line
import cmodule.ALFPackage.CLibInit;
| _hopSize | The hop between audio frames. This is also the number of samples read in on each frame, except the first frame where twice the number of samples are read |
| _STEREO | A boolean indicating whether the file is mono or stereo (False for mono, true for stereo) |
| _fs | The sample frequency |
| frameLookAhead | The number of frames that will be processed before audio playback begins |
public function setFrame( audio: ByteArray, type: String ):int
This function copies the data from a ByteArray into the shared Actionscript/C memory. The C flags are reset and then the C input buffer is polled to determine if it can accept more data without overrunning itself. If a write is deemed safe, the number of available samples is calculated and then written to the shared memory, the number of samples is also written to the memory for proper processing by C functions.
| audio | A ByteArray object containing raw sample data. For .wav files URLLoader can be used if passed as URLLoader.data. |
| type | A string specifying ‘float’ or ‘short’ datatype. |
Returns 1 if data was written, 0 if no data was written See Also:
DATF.setFrame(wavData.data, "short");
public function FFT( _fftSize: uint ):Array
Computes the Fast Fourier Transform (FFT) of the current frame.
| _fftSize | The nuber of DFT points to be used in calculating the Discrete Fourier Transform. Enter 0 for the default behavior. The default is the next power of two greater than the hopSize and is the recommneded way to compute the DFT. |
An array of alternating real and complex values.
public function IFFT( fftSize: uint ):Array
Performs an Inverse Fourier Transform on the current frame.
| fftSize | The number of IDFT points to be used in calculating the Inverse Fourier Transform. For reconstruction of a signal, this number needs to be the same as the fftSize used when the forward transform was calculated. |
An array of sample points.
public function magSpectrum( _fftSize: int, useDB: Number ):Array
Calculates the magnitude spectrum from the complex frequency spectrum. To calclate a DFT of another size other than the framSize, use FFT() first with the desired hopSize as a parameter and then call magSpec().
| _fftSize | The size of the FFT used in calculating the spectrum. A value of 0 sets this to the default, which is the greatest power of two higher than the frame size. |
| useDB | A value of 1 will return the magnitude spectrum in decibels, 0 for unmodified magnitude. |
public function getHarmonicAmplitudes():Array
A function to return the amplitude of the harmonics.
Returns an array containing the partial amplitudes populated by calling the getHarmonics function. These amplitudes are specified in decibels (dB). Note: getHarmonics() must be called before this function in order to populate the array with meaningful data.
public function getHarmonicFrequencies():Array
Returns an array containing the partial’s frequencies populated by callilng the getHarmonics function. The frequencies indicated are in Hertz (Hz). Note: getHarmonics() must be called before this function in order to return meaningful data.
public function getHarmonics( desiredHarms: uint ):void
Isolates the harmonics (or partials more generally speaking) of spectral data. The function operates on a particular channel pointer in order to extract the partials from the spectrum. getHarmonics populates two arrays: harmFreqs and harmAmps which contain the frequencies of the partials and their respective amplitudes (in dB). Separate calls are required to retrieve harmAmps (getHarmAmps) and harmFreqs (getHarmFreqs). This function must be called before the others to return relevant data.
Results will vary depending on a variety of factors, such as the type of spectrum (noisy or harmonic) as well as the hopSize used to process the audio. A harmonic-like spectrum should return the fundamental frequency as well as partials that are related by integer multiples of the fundamental (harmonics: i.e. 110Hz, 220Hz, 440Hz, ...etc). A noisy spectrum will yield non-harmoincally related peaks. Also, a large hopSize will tend to smear the spectral data, since the stationarity assumption does not hold over longer time windows.
The C functioncall returns an array indicating 1) if an error was encountered 2) the number of harmonics found (may be less than the number requested) 3) the pointer for the amplitude (peaks) array and 4) the pointer for the frequency array
| desiredHarms | An int specifying the desired number of harmonics the function should return. Int value should be: 0 < desiredHarms < (hopSize/2 + 1) |
public function getPitchFromSpeech( pitchMin: Number, pitchMax: Number, intenThreshold: Number ):Number
Same as getPitch except it provides min/max pitch values and a threshold useful for determining if there is an active signal in the current frame.
| pitchMin | the minimum acceptable pitch value |
| pitchMax | the maximum acceptable pitch value |
| intenThreshold | the intensity threshold value |
The estimated pitch value
public function addReverb( activate: String, level: Number, roomX: Number, roomY: Number, roomZ: Number, srcX: Number, srcY: Number, srcZ: Number, micX: Number, micY: Number, micZ: Number ):void
This function implements a well-known image model in order to simulate room reverb based on simulated room dimensions. A filtering method in the C methods implements a fast-convolution-based filter to add the required reverb. The reverb response is calculated based on the emitting-source’s position in the room, the room’s dimensions and the listener (microphone) position. By default the reverb is applied to both channels.
| activate | A String, either “on” or “off”. This sets the state of the reverb. Once turned “on”, it will remain on until turned off. If “off”, it will remain so until turned on. |
| level | A Number value that indicates the reflection strength. Possible values are integers in the range [1-4]. |
| roomX | the width of the simulated room in the x dimension |
| roomY | the width of the simulated room in the y dimension |
| roomZ | the height of the simulated room in the z dimension |
| srcX | the source (audio) position in the x dimension |
| srcY | the source (audio) position in the y dimension |
| srcZ | the source (audio) position in the z dimension |
| micX | the mic (listener) position in the x dimension |
| micY | the mic (listener) position in the y dimension |
| micZ | the mic (listener) position in the z dimension |
public function vocoder( active: Boolean, newPitch: Number, newTempo: Number )
Applies the phase vocoder functionality to each AudioChannel object unique to the DATF
| active | a boolean value that indicates if the vocoder is on (1) or off(0) |
| newPitch | a number (0.5 - 2.0) indicating the factor by which to modify the tempo. 0.5 correspoinds to an octave up and 2.0 is an octave below. |
| newTempo = a number (0.5 | 2.0) indicating the factor by which to modify the the tempo. 0.5 corresponds to twice the rate and 2.0 corresponds to half the rate. |
public function clearAudioBuffers():void
A Function to clean out the audioBuffers at the completion of audio file playback.
Buffers cleaned are -
public function checkOutputBuffer( channelType: String ):Array
A function to determine if the C-based audioFrame is ready for reading during audio playback.
| channelType | A string indicating the desired channel to be checked. “left” and “right” are valid arguments. |
An array containing
public function endOfFile():void
This function should be called when a file is complete or after ALF.stopAudio has been called. The buffers are cleared, flags reset, and parameters in the AudioChannel set accordingly to begin playback of the beginning of a file.
AudioChannel.cpp, <ALF>
public function getChOutPtrs():Array
Returns: An array containing the pointers to the C audio buffers. The first element is the left channel pointer, the second element is the right channel. Note these pointers are different from what is returned by getCRAM() in that they provide the location of the current sample in the audio buffers where as getCRAM() provides a pointer to the entire shared C memory buffer which includes all variables instantiated in <ALFPackace.cpp>.
public function reInitializeChannel( hop: uint, sampleRate: uint, channels: Number, frameLookAhead: Number ):void
This funciton should be called when a new song is loaded with a different sample rate(for wav) or the hop size is changed. This equates to a change in the number of frames per second. It will create a right channel if one does not exist.
| hop | The new hop size. |
| sampleRate | The new sample rate. |
| channels | The number of channels in the new song. |
| frameLookAhead | The offset (in frames) between the current data being returned and audio playback. |
AudioChannel.cpp->reInitChannel, ALFPackage.cpp->reInitializeChannel
Constructor to create a DATF object.
public function DATF( _hopSize: uint, _STEREO: Boolean, _fs: uint, frameLookAhead: Number ):void
This function copies the data from a ByteArray into the shared Actionscript/C memory.
public function setFrame( audio: ByteArray, type: String ):int
Computes the Fast Fourier Transform (FFT) of the current frame.
public function FFT( _fftSize: uint ):Array
Performs an Inverse Fourier Transform on the current frame.
public function IFFT( fftSize: uint ):Array
Calculates the magnitude spectrum from the complex frequency spectrum.
public function magSpectrum( _fftSize: int, useDB: Number ):Array
Calculates the spectral bandwidth for the current frame.
public function getBandwidth():Number
Calculates the spectral centroid for the current spectral data.
public function getCentroid():Number
Calculates the change in spectral energy between the current frame and the previous frame.
public function getFlux():Number
Calculates the spectral intensity for the current frame.
public function getIntensity():Number
Calculates the spectral rolloff for the current frame.
public function getRolloff():Number
A function to return the amplitude of the harmonics.
public function getHarmonicAmplitudes():Array
Returns an array containing the partial’s frequencies populated by callilng the getHarmonics function.
public function getHarmonicFrequencies():Array
Isolates the harmonics (or partials more generally speaking) of spectral data.
public function getHarmonics( desiredHarms: uint ):void
Calculates the linear prediction coefficients using Levinson-Durbin recursion.
public function getLPC( order: int ):Array
Uses the autocorrelation method to estimate the pitch of frame.
public function getPitch():Number
Same as getPitch except it provides min/max pitch values and a threshold useful for determining if there is an active signal in the current frame.
public function getPitchFromSpeech( pitchMin: Number, pitchMax: Number, intenThreshold: Number ):Number
This function implements a well-known image model in order to simulate room reverb based on simulated room dimensions.
public function addReverb( activate: String, level: Number, roomX: Number, roomY: Number, roomZ: Number, srcX: Number, srcY: Number, srcZ: Number, micX: Number, micY: Number, micZ: Number ):void
Filters the audio with the given filter (array of coefficients).
public function filterAudioFIR( coeffs: Array ):void
Filters the audio with the given filter (array of coefficients).
public function filterAudioIIR( numCoeffs: Array, denCoeffs: Array, gain: Number ):void
Applies the phase vocoder functionality to each AudioChannel object unique to the DATF
public function vocoder( active: Boolean, newPitch: Number, newTempo: Number )
A Function to clean out the audioBuffers at the completion of audio file playback.
public function clearAudioBuffers():void
A function to determine if the C-based audioFrame is ready for reading during audio playback.
public function checkOutputBuffer( channelType: String ):Array
This function should be called when a file is complete or after ALF.stopAudio has been called.
public function endOfFile():void
A function to stop playback of an audio file.
public function stopAudio():void
Returns: An array containing the pointers to the C audio buffers.
public function getChOutPtrs():Array
Returns: A pointer to the C memory buffer.
public function getCRAM():ByteArray
The current hopSize in the DATF.
public function getHopSize():uint
The current number of channels.
public function getNumberOfChannels():uint
This funciton should be called when a new song is loaded with a different sample rate(for wav) or the hop size is changed.
public function reInitializeChannel( hop: uint, sampleRate: uint, channels: Number, frameLookAhead: Number ):void
This resets all of the buffers and values to their state at initialization.
public function resetAll():void
Computes the autocorrelation of a given sequence, which is just its cross correlation with itself.
void autoCorr( float *corrData, int fftSize )
This funciton is called when a channel (L or R) has already been instantiated and initialized and a new song is loaded that requires changes to the buffer sizes.
void AudioChannel::reInitChannel( int _hopSize, int _fftSize, int _fs, int numCh, int _lookAheadFrames )
Re-initializes the channel after a new song of different sample rate has been loaded
AS3_Val reInitializeChannel( void* self, AS3_Val args )
This funciton resets all of the buffers in the AudioChannel as well as the flags.
AS3_Val resetAll( void *self, AS3_Val args )