If the song is yours and the only copy you can find is the one on YouTube, you can still get usable stems out of it. But the separator can’t give back what the codec already threw away. Before you split anything, hunt down the earliest-generation file you have, check that it really is lossless, decode it to WAV exactly once, and then split on your own machine. The kick and bass will survive a lossy copy fine. The cymbals and air won’t.

Most of us have been here. A track you made three laptops ago, the session folder gone with a dead drive, and the record still sitting on your channel with a few thousand plays. Or a live set your band filmed, you have the rights to it, and the multitrack never got exported. You want the vocal out of it for a remix, or the drums for a new edit, or just the bassline so you can finally figure out what you played.

Stem separation can do that now, and it does it well. What it can’t do is undo the trip your audio took to get to you. That trip is the part people skip, and it’s where most of the “AI stems sound watery” complaints actually start.

First, whose audio is it

This essay is about your own music, or music you’ve been given rights to. That isn’t a disclaimer bolted onto the end. It changes what you’re allowed to do and where you should look for the file. YouTube’s Terms of Service say you can’t download content except where the service expressly authorizes it, or with prior written permission from YouTube and the rights holders. Pulling someone else’s single off a video page so you can strip the vocal falls outside that, and the copyright problem follows you into anything you release.

For your own uploads there’s a sanctioned route. YouTube’s help page on downloading videos you’ve uploaded describes it: open YouTube Studio, go to Content, choose the video, and pick Download from its menu. You get an MP4 at 720p or 360p. Google Takeout can export everything you’ve uploaded in one go. That MP4 is your fallback, but it isn’t your first choice, for reasons the numbers below make clear.

What a separator is really doing

Modern separators like Demucs and the MDX-Net family learn, from thousands of multitrack songs, how to estimate each source from a mix. Many of them work, at least partly, by building a mask over a time-frequency picture of the song: for every small tile of time and frequency, how much of it belongs to the vocal, the drums, the bass. The rest is detail, and we go deep on the differences in the Demucs vs MDX-Net essay.

The detail that matters here is simpler. Whatever lands in those tiles gets handed to some stem. If a codec replaced the top octave of your hi-hats with smeared, quantized noise, the separator can’t tell that noise from cymbal. It just does its best with it. The source copy sets a ceiling, and no model, however good, gets above it.

Measuring the ceiling

We wanted to put a number on that, so we built a test with no real record in it at all. lab.py synthesizes a short 92 BPM groove: a pitch-swept kick, a band-limited saw bass, a detuned pad, a vowel-like lead standing in for a voice, and noise-burst hats. Because we made every stem ourselves, we know the truth exactly.

Then we push the mix through the lossy chains a real song tends to pass through: a single AAC or Opus encode, an MP3, and the classic “downloaded it, then a converter re-exported it” double hop. Every decoded copy gets split with the same oracle soft masks, built from the true stems. That’s the best any mask could ever do, a perfect separator. So any drop in quality is damage done by the copy, not by the separator.

#!/usr/bin/env python3
"""source_ceiling.py: how much separation quality a lossy copy costs you
before any AI model touches it.

SYNTHETIC test: a 92 BPM groove (kick, bass, pad, vowel-like lead, hats) built
in lab.py. We push the mix (as 2-channel, like a real stereo file) through common lossy chains with ffmpeg, then split
every decoded copy with the SAME oracle soft masks (computed from the true
stems, the best a mask can ever do). Any drop in SDR is damage the source copy
did, not the separator.

Usage: python3 source_ceiling.py            (needs numpy + ffmpeg)
"""
import numpy as np
import lab

N_FFT = 4096
stems = lab.synth_stems(seed=7)
mix, stems = lab.mixdown(stems)
masks = lab.ratio_masks(stems, N_FFT, power=2.0)

AAC = lambda k: ("m4a", ["-ac", "2", "-c:a", "aac", "-b:a", k])
MP3 = lambda k: ("mp3", ["-ac", "2", "-c:a", "libmp3lame", "-b:a", k])
OPUS = lambda k: ("opus", ["-ac", "2", "-c:a", "libopus", "-b:a", k])
chains = [
    ("lossless WAV (your bounce)", []),
    ("AAC 256k", [AAC("256k")]),
    ("AAC 128k", [AAC("128k")]),
    ("Opus 128k", [OPUS("128k")]),
    ("MP3 128k", [MP3("128k")]),
    ("AAC 128k -> MP3 192k", [AAC("128k"), MP3("192k")]),
    ("AAC 128k -> MP3 128k -> AAC 128k", [AAC("128k"), MP3("128k"), AAC("128k")]),
]

def hf_share(x, lo=15000):
    X = np.abs(np.fft.rfft(x)) ** 2
    f = np.fft.rfftfreq(len(x), 1 / lab.SR)
    return X[f >= lo].sum() / (X.sum() + 1e-20)

hf_ref = hf_share(mix)
names = list(stems)
print(f"synthetic mix: {len(mix)/lab.SR:.1f} s mono @ {lab.SR} Hz, oracle masks n_fft={N_FFT}")
print(f"{'source copy':36s} {'mix SNR':>8s} {'>15k kept':>9s} " +
      " ".join(f"{n:>6s}" for n in names) + f" {'mean':>6s}")
for label, chain in chains:
    if chain:
        y, lag = lab.codec_roundtrip(mix, chain)
    else:
        y, lag = mix.copy(), 0
    est = lab.separate_with_masks(y, masks, N_FFT)
    s = [lab.sdr(stems[n], est[n]) for n in names]
    snr = lab.sdr(mix, y)
    kept = 100 * hf_share(y) / hf_ref
    snr_txt = "   inf" if not chain else f"{snr:6.1f}"
    print(f"{label:36s} {snr_txt:>8s} {kept:8.0f}% " +
          " ".join(f"{v:6.1f}" for v in s) + f" {np.mean(s):6.1f}")
print("SDR in dB against the true synthetic stems (MDX-2021 definition). Higher is better.")

This is the real output from the box. The shared helpers (synthesis, STFT, SDR, codec round-trip with delay alignment) are in lab.py in the demos folder.

synthetic mix: 8.0 s mono @ 44100 Hz, oracle masks n_fft=4096
source copy                           mix SNR >15k kept   kick   bass    pad   lead   hats   mean
lossless WAV (your bounce)                inf      100%    9.6    9.1    9.1   16.9   18.1   12.6
AAC 256k                                 30.0       98%    9.6    9.0    9.1   16.9   13.7   11.7
AAC 128k                                 25.1       20%    9.6    9.0    9.0   16.9    2.4    9.4
Opus 128k                                23.7       65%    9.6    8.9    9.0   16.8    5.2    9.9
MP3 128k                                 23.0       20%    9.4    8.9    9.0   16.4    2.8    9.3
AAC 128k -> MP3 192k                     23.8       21%    9.5    9.0    9.0   16.7    2.4    9.3
AAC 128k -> MP3 128k -> AAC 128k         21.3       17%    9.4    8.8    8.9   16.2    1.5    9.0
SDR in dB against the true synthetic stems (MDX-2021 definition). Higher is better.

Reading it like an engineer

Look at the columns, not the averages. The kick, bass, pad and lead hardly notice. They live where codecs spend their bits, and at 128 kbps their stems stay within a few tenths of a dB of the lossless result. The hats tell the other story. They go from 18.1 dB SDR on the bounce to 2.4 dB through a single 128k AAC encode, and to 1.5 dB after three generations. At 128k on a stereo file, both AAC and MP3 kept only about a fifth of the energy above 15 kHz.

A 2–3 dB SDR stem is the swirly, sandy cymbal you’ve heard out of every quick YouTube split. The separator didn’t invent that. The copy handed it a top end that was mostly codec.

Two honest caveats. This is one synthetic groove in mono, run through ffmpeg’s encoders. YouTube’s own encoder settings aren’t public in a form we can reproduce, so read these as directions, not as a measurement of YouTube itself. And the kick’s ceiling of about 9.6 dB is a limit of the 4096-point mask, not the codec. Long analysis windows blur transients. That’s a story for the model comparison.

Find out what you actually have

The worst file in your archive is the one that says .wav and isn’t. A WAV decoded from an MP3 keeps the codec’s low-pass shelf forever, and nothing in the file name or the header will tell you. This little checker prints what ffprobe sees and then finds where the long-term spectrum drops off a cliff:

#!/usr/bin/env python3
"""fake_lossless_check.py: is this "WAV" really a first-generation file?

A WAV that was decoded from an MP3/AAC keeps the codec's low-pass shelf.
This prints the codec ffprobe sees, plus the frequency where the long-term
spectrum falls off a cliff. Usage: python3 fake_lossless_check.py file [file...]
With no arguments it builds SYNTHETIC test files from lab.py and checks those.
"""
import json, os, subprocess, sys, tempfile
import numpy as np
import lab

def probe(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "a:0", "-show_entries",
                          "stream=codec_name,sample_rate,bits_per_raw_sample,bit_rate",
                          "-of", "json", path], capture_output=True, text=True).stdout
    return json.loads(out)["streams"][0]

def cliff(path):
    x = lab.read_audio(path)
    n = 8192
    frames = [x[i:i + n] * np.hanning(n) for i in range(0, len(x) - n, n // 2)]
    P = np.mean([np.abs(np.fft.rfft(f)) ** 2 for f in frames], axis=0)
    f = np.fft.rfftfreq(n, 1 / lab.SR)
    PdB = 10 * np.log10(P + 1e-20)
    ref = np.median(PdB[(f > 2000) & (f < 8000)])
    # walk 250 Hz bands from 10 kHz up; the cliff is the last band within 30 dB of the mids
    top = 10000.0
    for lo in np.arange(10000, lab.SR / 2 - 250, 250):
        band = PdB[(f >= lo) & (f < lo + 250)]
        if band.size and band.mean() > ref - 30:
            top = lo + 250
    return top, None

def report(path):
    p = probe(path)
    top, shelf = cliff(path)
    verdict = "looks full-band" if top > 19500 else "LOW-PASS SHELF: probably decoded from a lossy file"
    print(f"{os.path.basename(path):28s} codec={p.get('codec_name'):10s} sr={p.get('sample_rate')} "
          f"content to ~{top/1000:4.1f} kHz  -> {verdict}")

if len(sys.argv) > 1:
    for pth in sys.argv[1:]:
        report(pth)
else:
    stems = lab.synth_stems(seed=7)
    mix, _ = lab.mixdown(stems)
    with tempfile.TemporaryDirectory() as d:
        wav = os.path.join(d, "bounce_24bit.wav"); lab.write_wav(wav, mix)
        mp3 = os.path.join(d, "tmp.mp3")
        subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", wav, "-ac", "2", "-c:a", "libmp3lame", "-b:a", "128k", mp3], check=True)
        fake = os.path.join(d, "fake_lossless_from_mp3.wav")
        subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", mp3, "-ac", "1", "-c:a", "pcm_s24le", fake], check=True)
        m4a = os.path.join(d, "tmp.m4a")
        subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", wav, "-ac", "2", "-c:a", "aac", "-b:a", "128k", m4a], check=True)
        fake2 = os.path.join(d, "fake_lossless_from_aac.wav")
        subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", m4a, "-ac", "1", "-c:a", "pcm_s24le", fake2], check=True)
        print("SYNTHETIC files (built from lab.py):")
        for pth in (wav, fake, fake2):
            report(pth)

SYNTHETIC files (built from lab.py):
bounce_24bit.wav             codec=pcm_s24le  sr=44100 content to ~22.0 kHz  -> looks full-band
fake_lossless_from_mp3.wav   codec=pcm_s24le  sr=44100 content to ~16.8 kHz  -> LOW-PASS SHELF: probably decoded from a lossy file
fake_lossless_from_aac.wav   codec=pcm_s24le  sr=44100 content to ~17.5 kHz  -> LOW-PASS SHELF: probably decoded from a lossy file

All three files say pcm_s24le, 44.1 kHz. The header has no idea. The spectrum knows: real content to 22 kHz on the true bounce, and a hard shelf around 17 kHz on the two that came from 128k lossy files. Run it on anything you find before you trust it. A shelf near 16–20 kHz means a lossy ancestor somewhere.

The order that gets you the best stems

1. Dig for the first generation. Check old bounce folders and external drives. Search your sent mail for the WAV you sent the mastering engineer. Look at your distributor’s dashboard, since some keep the file you delivered. Ask collaborators and the studio that tracked you. A 16-bit master from 2014 beats a 2024 upload every time.
2. If YouTube really is all that’s left, use the official download. YouTube Studio for your own channel, Takeout for the whole archive. For a band or label channel, ask whoever controls it. Don’t point a ripper at a video you don’t control.
3. Decode once, to WAV, and archive that file. ffmpeg -i upload.mp4 -vn -c:a pcm_s24le song.wav. Converting doesn’t add quality back. It stops the next tool from encoding it again. Every “export as MP3 to save space” in the chain is another row down that table.
4. Ask for the fewest stems that do the job. Vocals, drums, bass and other is the most mature split in published benchmarks. Separate guitar and piano stems are handy, but similar timbres bleed into each other more.
5. Split locally, audition, then null-test. Sum the stems back together and subtract the original. The leftover tells you what the separator dropped or invented, and it’s often a quick read on how much the source copy hurt.
6. Export stems as WAV. They’re working files. Compress the delivery, never the parts.

For the splitting step, a local tool keeps the whole thing in your room. DeepSplit runs Demucs- and MDX-style models on your own computer, from two stems up to drum sub-separation, and writes lossless stems. If you’re working from an upload of your own unreleased demo, that matters. The file never has to go to someone else’s server just to be taken apart.

When the lossy copy is all you’ll ever have

Sometimes the master is truly gone. Then you’re working with the stems the copy allows, and a few habits help.

• Trust the low and mid stems. Vocal, bass, kick and most of the snare body came through our test nearly untouched. Build the remix around those.
• Replace, don’t rescue, the top end. Swirly hats are codec damage. Re-program them, or layer a clean shaker or ride over a filtered version of the original, instead of boosting 12 kHz on something that’s mostly noise.
• Don’t high-shelf the vocal to “restore air.” You’ll be turning up quantization noise. A gentle exciter driven from the 3–6 kHz region tends to sound more honest than EQ above the shelf.
• Measure after you rebuild. A remix built from separated parts still has to hit the same delivery numbers. Re-encoding it on the way out costs peak headroom again, as Re-encoding eats true-peak headroom shows. Leave margin for it.

When the new version is printed, a pass through CoProducer’s analyzer is a good last look: loudness, true peak and the stereo picture, measured on your own machine, before the file heads back to the platform it came from.

Is it legal to split stems from a YouTube video?

It depends on whose audio it is. YouTube’s Terms don’t allow downloading content unless the service expressly authorizes it or you have permission from YouTube and the rights holders. For your own uploads, YouTube Studio offers a download. For anyone else’s song, get permission or a licensed copy first.

How do I download a video I uploaded myself?

Sign in to YouTube Studio, open Content, pick the video, and choose Download from its menu. YouTube’s help page says you get an MP4 at 720p or 360p. Google Takeout can export all your uploads at once.

Why do my separated hi-hats sound swirly?

Most of the time it’s the source copy. Lossy codecs drop or coarsen the top octave, and the separator then has to assign codec noise to some stem. In our synthetic test the hats lost most of their SDR at 128 kbps while kick and bass barely moved.

Will converting the MP4 audio to WAV improve the stems?

No. Converting to WAV only stops further damage. Decode once to WAV and work from that file so you don’t stack another lossy generation on top.

How many stems should I ask for?

Ask for the fewest that do the job. Four stems (vocals, drums, bass, other) are the most mature in published benchmarks. Extra guitar or piano stems are useful, but expect more bleed between similar instruments.