Hack 75. Build an Audio Waveform Display

TheendresultofthishackisdisplayedinFigure10-5.You'll

startbyreadingintheentireaudiofileusinganAudioInputStream.

Thenyou'llconverttherawdatafromthestreamintouseful

audiosamples,organizedbychannel.Withtheconverted

channelaudiodata,you'llcreateasinglewaveformpanel.Then

you'llwrapupthecompleteaudiodisplaybycombiningseveral

waveformpanelstodisplaymulti-channelaudio.



Figure10-5.Thewaveformdisplayyou'llbuildin

thishack



10.7.1.SomeBasicDefinitions

You'llneedtoknowafewbasictermsandconceptsaboutaudio

beforeyougetstarted.



Sample

Onemeasurementofaudiodata.ForPulseCodeModulated

(PCM)encoding,asampleisaninstantaneous

representationofthevoltageoftheanalogaudio.Thereare



othertypesofencoding,like -lawanda-law,thatare

rarelyused.



SamplingRate

Thenumberofsamplesinonesecond.MeasuredinHertz

(Hz)orkilo-Hertz(kHz).Themostcommonsamplingrateis

44.1kHz(CDqualityaudio).Often,you'llfind22.05kHzor

11.025kHzontheWeb,sincethefilesaresmallerandthe

conversioniseasier.



SampleSize

Thenumberofbitsinonesample.Itistypicallyamultiple

ofeightbecausedataisstoredin8-bitbytes.Themost

commonsamplesizeis16bits,whichisCDqualityaudio.

Oftenyou'llfind8-bitaudiobecausethefilesaresmaller.

You'llrarelyfindanythinglessthen8-bitaudiobecausethe

qualityisprettypoor.Samplesizeissometimescalledbit

depth.



Channel

Achannelisanindependentstreamofaudio.Stereoisthe

mostcommonformofmulti-channelaudiooneindependent

leftandrightchannel.Higher-endaudioformatsinclude5.1

surroundsound(actuallysixchannels)andup.



Frame

Aframeisacrosssectionofsamplesacrossallchannelsin



theaudiofile.So,a16-bitstereo(twochannel)audiofile

willhave32-bitframes(16bitspersample*2channelsper

frame=32bitsperframe).



10.7.2.LoadtheRawData

Javareadsrawaudiodatain8-bitbytes,butmostaudiohasa

highersamplesize.So,inordertorepresenttheaudio,you'll

havetocombinemultiplebytestocreatesamplesintheaudio

format.Butfirst,you'llneedtoloadalloftheaudiointoabuffer

beforeyoucombinethebytesintosamples.

Startbygettinganaudiostreamfromafile:







Filefile=newFile(filename);

AudioInputStreamaudioInputStream=AudioSystem.getAudi



NowthatyouhavetheAudioInputStream,youcanreadintheaudio

data.AudioInputStreamhasaread()methodthattakesan

unpopulatedbyte[]andreadsindatathelengthofthebyte[].To

readintheentireaudiofileinoneshot,createabyte[]the

lengthoftheentireaudiofile.Thecompletelengthofthefilein

bytesis:





totalnumberofbytes=bytesperframe*totalnumber



Youcangetthenumberofframesforthewholefile(frameLength)

andthesizeoftheframe(frameSize)fromtheAudioInputStream:







intframeLength=(int)audioInputStream.getFrameLength

intframeSize=(int)audioInputStream.getFormat().getF



Youcancreatethebyte[]withthelengthsetto

frameLength*frameSize:





byte[]bytes=newbyte[frameLength*frameSize];



Finally,youcanreadintheaudio,passingtheAudioInputStream

theemptybyte[]andcatchingtheappropriateexceptions:















intresult=0;

try{



result=audioInputStream.read(bytes);

}catch(Exceptione){



e.printStackTrace();

}



10.7.3.ConverttoSamplesandChannels

Therawaudiodataisn'tveryuseful.Itneedstobebrokenup

intochannelsandsamples.Fromthere,it'seasytopaintthe

samples.

Thebyteswillbeconvertedtosamplesandrepresentedasints.

You'llneedacontainertostorethesamplesacrossallchannels.

So,createatwodimensionalint[][]referencingthechannel

andsamplesperchannel.You'vealreadyseenhowtogetthe

framelengthfromtheAuduioInputStream,andyoucangetthe

numberofchannelsthesameway.Hereisthecodetoinitialize

theint[][]:

intnumChannels=audioInputStream.getFormat().getChannels();



intframeLength=(int)audioInputStream.getFrameLength();

int[][]toReturn=newint[numChannels][frameLength];



Now,youneedtoiteratethroughthebyte[],convertthebytes

tosamples,andplacethesampleintheappropriatechannelin

theint[][].Thebyte[]isorganizedbyframes,meaningthat

you'llreadinasampleforeverychannelratherthanallofthe

samplesforaspecificchannelinarow.So,theflowistoloop

throughthechannelsandaddsamplesuntilthebyte[]hasbeen

iteratedcompletely:

intsampleIndex=0;



for(intt=0;t


for(intchannel=0;channel




intlow=(int)eightBitByteArray[t];





t++;





inthigh=(int)eightBitByteArray[t];





t++;





intsample=getSixteenBitSample(high,low);





toReturn[channel][sampleIndex]=sample;



}



sampleIndex++;

}



Thishackisgoingtodealexclusivelywith16-bitsamples.Theyareby

farthemostcommon.Plus,youcangetanideaforhowsample

conversionworkswhilestillkeepingthingsprettystraightforward.This

codegetsmuchtrickierwithmultipledynamicsamplesizes.



NowforthegetSixteenBitSample()method.Youcan'tsimplyadd

thebytestogetherusingregularadditionbecausethebitsare

displacedina16-bitsamplethehighbyterepresentsbits0

through7,andthelowbyterepresentsbits8through15.It's

morelikeconcatenation,sothetypeofmathshownherewon't

work:



10101101(highbyte)

+00110010(lowbyte)

11011111



Whatyouwantismorelikethis:



10101101



(highbyte)

+



00110010(lowbyte)

--------------------1010110100110010



Andinordertogetthistoworkwithstandardaddition,you

needtoaddtwo16-bitbyteswithbitsshiftedandplaceholder

0saddedwherenecessary.Thenyougetsomethinglikethis:



1010110100000000(highbyte)

+0000000000110010(lowbyte)

--------------------1010110100110010



Thehighbyteneedstobebitshifted.Bitshifting,theprocessof

slidingbitsaround,istypicallyabigno-noinJavaasaresult,

you'veprobablyneverseenthebit-shiftingoperatorbefore(it's

<>dependingonthedirectionfollowedbythenumberof



bitstoshiftineitherdirection).However,hereitisnecessaryto

usebitshifting,soyouwillbitshiftthehighbyte8bitstothe

left:

high<<8



Now,youneedtoprependtheleading0sontothelowbyte.You

candothisusingthebitANDoperatorandusinga16-bitbyte

consistingofall0s.Itworkslikethis:



0000000000000000(all0'sbytes)

+



00110010(lowbyte)

--------------------0000000000110010



Hereisthecodeforthesampleconversion:









privateintgetSixteenBitSample(inthigh,intlow){



return(high<<8)+(low&0x00ff);

}



10.7.4.CreatingaSingleWaveformDisplay

Nowthatyouhavetheaudiosampledataorganizedby

channels,it'stimetogettopainting.Tokeepeverything

modular,createaclasscalledSingleWaveformPaneltopaintone

channelofaudiodata.Inthenextsection,you'llwritea

WaveformPanelContainertousemultipleSingleWaveformPanelstohandle

multi-channelaudio.



Thewaveformpaintingisgoingtobedrawnbyplottingpoints

scaledtothesampledataanddrawinglinesbetweenthem.This

issimplistic,butityieldsgoodresults.Figures10-4and10-5

showthesamewaveforminAudacityandthesimulatorforthis

hack;they'reprettyclose.

I'mgoingtoglossoverthescalingcodebecauseIreallywant

toconcentrateontheconversionfromaudioinformationto

visualization.Buttounderstandwhyscalingisnecessary,

rememberthatCDqualityaudiohas44,100samplesper

second.So,withoutscaling,youwouldneed44,100horizontal

pixelsforeverysecondofyouraudiofile.Obviously,thisis

impractical.So,ifyoudigintothesourcecodeforthishack,

youcanseethescalingandhowthescalesaredetermined.

Meanwhile,justassumethatthewaveformisalwaysscaledto

fitinthepanel.

Startbydrawingthecenterlineat0:







g.setColor(REFERENCE_LINE_COLOR);

g.drawLine(0,lineHeight,(int)getWidth(),lineHeight);



Next,marktheorigintostartdrawingat0,0:









intoldX=0;

intoldY=(int)(getHeight()/2);

intxIndex=0;



Now,youneedtofigureouttheincrementaljumpbetween

samplestoadjustforthescalefactor.Thisworksouttobe:





numberofsamples/(numberofsamples*horizontalsca



Thefollowingcodegrabstheincrementandpaintsalinefrom

theorigintothefirstsample:







intincrement=getIncrement()

g.setColor(WAVEFORM_COLOR);







intt=0;















for(t=0;t


g.drawLine(oldX,oldY,xIndex,oldY);



xIndex++;



oldX=xIndex;

}



Finishupbyiteratingthroughtheaudioanddrawinglinestothe

scaledsamples:













for(;t


doublescaleFactor=getYScaleFactor();



doublescaledSample=samples[t]*scaleFactor;



inty=(int)((getHeight()/2)-(scaledSampl



g.drawLine(oldX,oldY,xIndex,y);























}



xIndex++;

oldX=xIndex;

oldY=y;

}



10.7.5.CreateaContainer

Nowthatyouhavethewaveformpaintingundercontrol,you



needtocreateacontainercalledWaveformPanelContainerfor

SingleWaveformPanelsinordertoshowmulti-channelaudio.Figure

10-6showsthewaveforminthesimulator.



Figure10-6.Multi-channel(stereo)audiointhe

simulatorforthishack



Example10-7isthecompletecodefortheWaveformPanelContainer.

AudioInfoisahelperclassthatcontainsreferencestotheloaded

audiosamplesandthecurrentchannel.



Example10-7.Testingoutthewaveformdisplay



publicclassWaveformPanelContainerextendsJPanel{



privateArrayListsingleChannelWaveformPanels=newArr



privateAudioInfoaudioInfo=null;





publicWaveformPanelContainer(){





setLayout(newGridLayout(0,1));



}







publicvoidsetAudioToDisplay(AudioInputStreamaudioInp



singleChannelWaveformPanels=newArrayList();





























audioInfo=newAudioInfo(audioInputStream);



for(intt=0;t




SingleWaveformPanelwaveformPanel







=newSingleWaveformPanel(audio





singleChannelWaveformPanels.add(wavefor





add(createChannelDisplay(waveformPanel,



}

}

privateJComponentcreateChannelDisplay(





SingleWaveformPanelwaveformPanel,





intindex){



JPanelpanel=newJPanel(newBorderLayout());



panel.add(waveformPanel,BorderLayout.CENTER);







JLabellabel=newJLabel("Channel"+++index);

panel.add(label,BorderLayout.NORTH);







}



returnpanel;

}



10.7.6.SeeingIsBelieving

Now,you'rereadytorunthehack.Themain()methodshown

hereisthesimulatorcode.Noticethecreationofthe

AudioInputStreamandthecreationofthecontainerwiththe

stream.AllpaintingandmanagementofSingleWaveformPanelsis

encapsulatedwithintheseparatepanelclasses:







publicstaticvoidmain(String[]args){



try{