Playing audio with speakers (2024)

CC: Tweaked's speaker peripheral provides a powerful way to play any audio you like with the speaker.playAudiomethod. However, for people unfamiliar with digital audio, it's not the most intuitive thing to use. This guide providesan introduction to digital audio, demonstrates how to play music with CC: Tweaked's speakers, and then briefly discussesthe more complex topic of audio processing.

A short introduction to digital audio

When sound is recorded it is captured as an analogue signal, effectively the electrical version of a soundwave. However, this signal is continuous, and so can't be used directly by a computer. Instead, we measure (or sample)the amplitude of the wave many times a second and then quantise that amplitude, rounding it to the nearestrepresentable value.

This representation of sound - a long, uniformally sampled list of amplitudes is referred to as Pulse-codeModulation (PCM). PCM can be thought of as the "standard" audio format, as it's incredibly easy to work with. Forinstance, to mix two pieces of audio together, you can just add samples from the two tracks together and take the average.

CC: Tweaked's speakers also work with PCM audio. It plays back 48,000 samples a second, where each sample is an integerbetween -128 and 127. This is more commonly referred to as 48kHz and an 8-bit resolution.

Let's now look at a quick example. We're going to generate a Sine Wave at 220Hz, which sounds like a low monotonoushum. First we wrap our speaker peripheral, and then we fill a table (also referred to as a buffer) with 128×1024samples - this is the maximum number of samples a speaker can accept in one go.

In order to fill this buffer, we need to do a little maths. We want to play 220 sine waves each second, where each sinewave completes a full oscillation in 2π "units". This means one seconds worth of audio is 2×π×220 "units" long. We thenneed to split this into 48k samples, basically meaning for each sample we move 2×π×220/48k "along" the sine curve.

local speaker = peripheral.find("speaker")local buffer = {}local t, dt = 0, 2 * math.pi * 220 / 48000for i = 1, 128 * 1024 do buffer[i] = math.floor(math.sin(t) * 127) t = (t + dt) % (math.pi * 2)endspeaker.playAudio(buffer)

Streaming audio

You might notice that the above snippet only generates a short bit of audio - 2.7s seconds to be precise. While we couldtry increasing the number of loop iterations, we'll get an error when we try to play it through the speaker: the soundbuffer is too large for it to handle.

Our 2.7 seconds of audio is stored in a table with over 130 thousand elements. If we wanted to play a full minute ofsine waves (and why wouldn't you?), you'd need a table with almost 3 million. Suddenly you find these numbers addingup very quickly, and these tables take up more and more memory.

Instead of building our entire song (well, sine wave) in one go, we can produce it in small batches, each of which getpassed off to speaker.playAudio when the time is right. This allows us to build a stream of audio, where we readchunks of audio one at a time (either from a file or a tone generator like above), do some optional processing to eachone, and then play them.

Let's adapt our example from above to do that instead.

local speaker = peripheral.find("speaker")local t, dt = 0, 2 * math.pi * 220 / 48000while true do local buffer = {} for i = 1, 16 * 1024 * 8 do buffer[i] = math.floor(math.sin(t) * 127) t = (t + dt) % (math.pi * 2) end while not speaker.playAudio(buffer) do os.pullEvent("speaker_audio_empty") endend

It looks pretty similar to before, aside from we've wrapped the generation and playing code in a while loop, and added arather odd loop with speaker.playAudio and os.pullEvent.

Let's talk about this loop, why do we need to keep calling speaker.playAudio? Remember that what we're trying to dohere is avoid keeping too much audio in memory at once. However, if we're generating audio quicker than the speakers canplay it, we're not helping at all - all this audio is still hanging around waiting to be played!

In order to avoid this, the speaker rejects any new chunks of audio if its backlog is too large. When this happens,speaker.playAudio returns false. Once enough audio has played, and the backlog has been reduced, aspeaker_audio_empty event is queued, and we can try to play our chunk once more.

Storing audio

PCM is a fantastic way of representing audio when we want to manipulate it, but it's not very efficient when we want tostore it to disk. Compare the size of a WAV file (which uses PCM) to an equivalent MP3, it's often 5 times the size.Instead, we store audio in special formats (or codecs) and then convert them to PCM when we need to do processing onthem.

Modern audio codecs use some incredibly impressive techniques to compress the audio as much as possible while preservingsound quality. However, due to CC: Tweaked's limited processing power, it's not really possible to use these from yourcomputer. Instead, we need something much simpler.

DFPWM (Dynamic Filter Pulse Width Modulation) is the de facto standard audio format of the ComputerCraft (andOpenComputers) world. Originally popularised by the addon mod Computronics, CC:T now has built-in support for it withthe cc.audio.dfpwm module. This allows you to read DFPWM files from disk, decode them to PCM, and then play themusing the speaker.

Let's dive in with an example, and we'll explain things afterwards:

local dfpwm = require("cc.audio.dfpwm")local speaker = peripheral.find("speaker")local decoder = dfpwm.make_decoder()for chunk in io.lines("data/example.dfpwm", 16 * 1024) do local buffer = decoder(chunk) while not speaker.playAudio(buffer) do os.pullEvent("speaker_audio_empty") endend

Once again, we see the speaker.playAudio/speaker_audio_empty loop. However, the rest of the program is a littledifferent.

First, we require the dfpwm module and call cc.audio.dfpwm.make_decoder to construct a new decoder. This decoderaccepts blocks of DFPWM data and converts it to a list of 8-bit amplitudes, which we can then play with our speaker.

As mentioned above, speaker.playAudio accepts at most 128×1024 samples in one go. DFPWM uses a single bit for eachsample, which means we want to process our audio in chunks of 16×1024 bytes (16KiB). In order to do this, we useio.lines, which provides a nice way to loop over chunks of a file. You can of course just use fs.open andfs.ReadHandle.read if you prefer.

Processing audio

As mentioned near the beginning of this guide, PCM audio is pretty easy to work with as it's just a list of amplitudes.You can mix together samples from different streams by adding their amplitudes, change the rate of playback by removingsamples, etc...

Let's put together a small demonstration here. We're going to add a small delay effect to the song above, so that youhear a faint echo a second and a half later.

In order to do this, we'll follow a format similar to the previous example, decoding the audio and then playing it.However, we'll also add some new logic between those two steps, which loops over every sample in our chunk of audio, andadds the sample from 1.5 seconds ago to it.

For this, we'll need to keep track of the last 72k samples - exactly 1.5 seconds worth of audio. We can do this using aRing Buffer, which helps makes things a little more efficient.

local dfpwm = require("cc.audio.dfpwm")local speaker = peripheral.find("speaker")-- Speakers play at 48kHz, so 1.5 seconds is 72k samples. We first fill our buffer-- with 0s, as there's nothing to echo at the start of the track!local samples_i, samples_n = 1, 48000 * 1.5local samples = {}for i = 1, samples_n do samples[i] = 0 endlocal decoder = dfpwm.make_decoder()for chunk in io.lines("data/example.dfpwm", 16 * 1024) do local buffer = decoder(chunk) for i = 1, #buffer do local original_value = buffer[i] -- Replace this sample with its current amplitude plus the amplitude from 1.5 seconds ago. -- We scale both to ensure the resulting value is still between -128 and 127. buffer[i] = original_value * 0.6 + samples[samples_i] * 0.4 -- Now store the current sample, and move the "head" of our ring buffer forward one place. samples[samples_i] = original_value samples_i = samples_i + 1 if samples_i > samples_n then samples_i = 1 end end while not speaker.playAudio(buffer) do os.pullEvent("speaker_audio_empty") end -- The audio processing above can be quite slow and preparing the first batch of audio -- may timeout the computer. We sleep to avoid this. -- There's definitely better ways of handling this - this is just an example! sleep(0.05)end

It's worth noting that the examples of audio processing we've mentioned here are about manipulating the amplitude ofthe wave. If you wanted to modify the frequency (for instance, shifting the pitch), things get rather more complex.For this, you'd need to use the Fast Fourier transform to convert the stream of amplitudes to frequencies,process those, and then convert them back to amplitudes.

This is, I'm afraid, left as an exercise to the reader.

See also

• speaker.playAudio Play PCM audio using a speaker.
• cc.audio.dfpwm Provides utilities for encoding and decoding DFPWM files.