QWEN × FISHDACRESEARCH NOTE / UPDATED 07 NOV 2026
APPROACH B · SINGLE-SPEAKER STUDY

A voice,
reconstructed.

One clip. Both reconstructions.
Compare a frozen Qwen decoder prefix with its unfrozen continuation, through the same FishDAC synthesis stack.

Enter the listening room ↗
PHIL BENSON / SPEAKER 6097Loading verified snapshot…
CHECKPOINT—Completed updates
TRAINING CORPUS29.95 h39,373 clean utterances · one speaker
OUTPUT SAMPLE RATE44.1 kHzOriginal amplitude · float WAV
LISTENING EXAMPLES6616 training + all 50 validation clips
THE SNAPSHOT

Loading measured results…

↓

NEW / COMPLETED 5,000-UPDATE ABLATION

Does unfreezing help?

Both branches started from the same full checkpoint at 73,179 and finished at 78,179. All 5,000 batches matched on every GPU. Only one branch unfroze the retained Qwen decoder pre-convolution, full 8-block transformer, and first ×2 upsampler + ConvNeXt (42.5M parameters; LR 1e-5). Encoder, codebooks and ASR teacher stayed frozen. Bridge, FishDAC and discriminator kept their existing learning rates and losses.

Held-out validation · all 50 clips · lower is better
MetricFrozen controlUnfrozen prefixUnfrozen vs frozen

The result is mixed: validation mel is nearly unchanged, while MR-STFT is slightly worse with unfreezing. This short single-seed test does not establish a perceptual improvement. Compare the audio below. The older 69,950 snapshot is preserved as a historical reference, not the matched control.

01 / LISTEN CLOSELY

The listening room

Matched utterances. No loudness normalization.
Frozen and unfrozen together for every clip.

HELD-OUT VALIDATION
SOURCE TRANSCRIPT
Loading…

Ground truth

A / ORIGINAL RECORDING↓ WAV
0:00 / 0:00

Frozen Qwen prefix step 78,179

Frozen · 44.1 kHz

MATCHED CONTROL↓ WAV
0:00 / 0:00

Frozen · 24 kHz

BAND-LIMITED CONTROL↓ WAV
0:00 / 0:00

Unfrozen Qwen prefix step 78,179

Unfrozen · 44.1 kHz

DECODER PREFIX LR 1e-5↓ WAV
0:00 / 0:00

Unfrozen · 24 kHz

BAND-LIMITED CONTROL↓ WAV
0:00 / 0:00

Compare both reconstructions

Each branch has its own 44.1-kHz output and the same output downsampled to 24 kHz. Both mel figures use identical time, frequency and color scales. The 24-kHz controls are not separate models.

Frozen prefix

frozen branch: ground truth, full-band reconstruction and 24-kHz control
↓ Download frozen spectrogram

Unfrozen prefix

unfrozen branch: ground truth, full-band reconstruction and 24-kHz control
↓ Download unfrozen spectrogram

Fixed −100 to −20 dB colors, no per-panel normalization. Dashed line: 12 kHz. High-band energy is not proof of faithful detail.

Inspect the signal Waveforms, spectrum & source ↗

Welch power spectral density, 2,048-sample windows. Mint: ground truth; lavender: frozen; amber: unfrozen; muted traces: their 24-kHz controls. All use the same axes; this is not a perceptual quality score.

Waveform envelopes use a shared peak scale within each original branch triplet. No gain normalization or fitted time shift is applied.

Validation = official dev.clean, excluded from this fine-tuning run. Training examples are a fixed seeded panel, not the full training set. The 100 official test clips are not used here.

02 / FOLLOW THE FIT

A trajectory, not a verdict.

Unsmoothed, fixed-panel evaluation.
Lower is better for both reconstruction losses.

Frozen (lavender) / unfrozen (amber) Solid: validation 50 · dashed: train 16x = additional updates from shared parent 73,179

Mel-spectrogram loss

Multi-resolution STFT loss

Each point is a per-clip mean on the same 16 train / 50 validation utterances. These are evaluation losses—not minibatch training curves, MOS, or proof of perceptual equivalence. Use the audio to judge residual artifacts.

03 / WHAT YOU ARE HEARING

Frozen encoder.
Adaptable synthesis.

The full utterance passes through the frozen Qwen encoder and the retained beginning of its native decoder. That decoder prefix is frozen in the control and trainable in the new ablation. A trainable bridge then conditions FishDAC’s synthesis stack.

This is a single-speaker fitting study, not an unseen-speaker benchmark. Validation shares the speaker and source books with training. Upstream pretraining overlap has not been audited.
Retained Qwen path
Always-frozen encoder + RVQ → decoder pre-convolution → full 8-block pre-transformer → first ×2 upsampler + ConvNeXt.
The bridge
1,024 features @ 25 Hz → fixed rate conversion to ≈21.53 Hz → trainable adapter.
Trainable FishDAC
Both latent ×2 upsamplers + ConvNeXt, then the waveform decoder → 44.1 kHz. Fish post-transformer bypassed.
Fine-tuning
Warm-started from B step 10,950, including G/D optimizer state. Global batch 8; constant LR: bridge 2e-4, synthesis 5e-5, discriminator 2e-5. The original run was stopped at 73,179; both ablation branches then ran exactly 5,000 updates. Only the unfrozen branch adds Qwen decoder LR 1e-5.
Objective
Mel + MR-STFT + GAN + feature matching + frozen Qwen-ASR perceptual loss. Native gain, no data augmentation.
Evaluation
Fresh full-utterance inference from three fixed checkpoints. Each snapshot provides 66 matched original, full-band reconstruction and 24-kHz control triplets. Fresh losses verified against saved evaluation.

OPEN THE EVIDENCE

Reproducible by design.

Data: original Hi-Fi TTS / OpenSLR 109, NVIDIA et al., CC BY 4.0; speaker 6097, clean partition. Ground-truth WAVs preserve the original samples. Reconstructions are experimental model outputs. Original chapter recordings were MP3-sourced; 44.1 kHz does not imply pristine full-band originals.

↓ Frozen metrics & hashes JSON↓ Unfrozen metrics & hashes JSONEarlier 69,950 report ↗↓ Method & reproduction MD
Snapshot identity & limitations

Speaker selection used recording-quality proxies and development audio. No MOS study or blind listening study is claimed. This is a static report snapshot; both ablation branches are complete and training is stopped.