grotsoV commited on
Commit
8f46b2e
·
verified ·
1 Parent(s): 7465e8c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -3
README.md CHANGED
@@ -1,3 +1,76 @@
1
- ---
2
- license: cc-by-nc-sa-4.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ tags:
4
+ - audio
5
+ - vocoder
6
+ - deepfake-enhancement
7
+ - synthetic-audio
8
+ - vocos
9
+ datasets:
10
+ - grotsoV/4TTSDeepfakeData
11
+ language:
12
+ - en
13
+ base_model:
14
+ - charactr/vocos-mel-24khz
15
+ ---
16
+
17
+ # Vocos for Deepfake Audio Enhancement
18
+
19
+ ## Model Description
20
+ This model is a custom-trained checkpoint of the [Vocos](https://github.com/charactr-platform/vocos) vocoder architecture, specifically fine-tuned to enhance the phase coherence and acoustic quality of synthetic speech (deepfakes).
21
+
22
+ It was trained on the [4TTSDeepfakeDataset](https://huggingface.co/datasets/grotsoV/4TTSDeepfakeData), which contains 1:1 parallel bona fide and zero-shot deepfake utterances generated by F5-TTS, MaskGCT, CosyVoice2, and Fish-S1mini.
23
+
24
+ * **Architecture:** Vocos (Mel-spectrogram to 24kHz waveform)
25
+ * **Sampling Rate:** 24,000 Hz
26
+ * **Primary Task:** Audio enhancement / Deepfake artifact reduction
27
+
28
+ ## Intended Use & Limitations
29
+ This model is released strictly for **academic research, educational, and personal purposes**.
30
+ It is intended to be used in audio forensics, deepfake detection research, and synthetic audio analysis pipelines.
31
+
32
+ **Prohibited Uses:** Due to the CC BY-NC-SA 4.0 license, you may **not** use these weights for any commercial purposes, productization, or malicious voice cloning applications.
33
+
34
+ ## How to Use
35
+ You can load these weights using the original `vocos` Python library. Because this is a fine-tune of the original 24kHz mel-spectrogram model, you instantiate the base model first and then load your downloaded weights.
36
+
37
+ ```bash
38
+ pip install vocos
39
+ ```
40
+
41
+ ```python
42
+ import torch
43
+ import torchaudio
44
+ from vocos import Vocos
45
+
46
+ # 1. Initialize the base Vocos model
47
+ vocos = Vocos.from_pretrained("charactr/vocos-mel-24khz")
48
+
49
+ # 2. Load the custom deepfake enhancement weights
50
+ # (Replace 'model.pt' with the actual name of the file you upload here)
51
+ state_dict = torch.load("model.pt", map_location="cpu")
52
+
53
+ # Handle nested state dictionaries if necessary
54
+ if 'vocos_state_dict' in state_dict:
55
+ vocos.load_state_dict(state_dict['vocos_state_dict'])
56
+ else:
57
+ vocos.load_state_dict(state_dict)
58
+
59
+ vocos.eval()
60
+
61
+ # 3. Inference example
62
+ wav, sr = torchaudio.load("path/to/your/deepfake.wav")
63
+ if sr != 24000:
64
+ wav = torchaudio.transforms.Resample(sr, 24000)(wav)
65
+
66
+ with torch.no_grad():
67
+ mel = vocos.feature_extractor(wav)
68
+ x = vocos.backbone(mel)
69
+ enhanced_wav = vocos.head(x)
70
+
71
+ torchaudio.save("enhanced_output.wav", enhanced_wav.cpu(), 24000)
72
+ ```
73
+
74
+ ## Acknowledgements and Attribution
75
+ * **Architecture:** The original Vocos model was created by [Charactr](https://charactr.com/) (Licensed under MIT).
76
+ * **Training Data:** Trained on the [4TTSDeepfakeDataset](https://huggingface.co/datasets/grotsoV/4TTSDeepfakeData), which is derived from the Hi-Fi TTS Dataset (CC BY 4.0) and utilizes outputs from F5-TTS, MaskGCT, Fish-S1mini, and CosyVoice2.