Gemma 3 4B - Cebuano, Ilocano, Tagalog Fine-tuning

A specialized fine-tuned version of Google's Gemma 3 4B optimized for low-resource Philippine languages: Cebuano, Ilocano, and Tagalog (Filipino).

Model Details

Model ID: nielle003/Gemma_3_4B_cebuano_ilocano_tagalog

Base Model: Google Gemma 3 4B License: Gemma Language: Cebuano, Ilocano, Tagalog (Filipino) Task: Instruction following, Question answering, Conversation

Training Data

Dataset Statistics

• Total Training Samples: 30,000
  • Training Set: 18,000 samples (7,500 real + 10,500 synthetic)
  • Validation Set: 9,000 samples (4,500 real + 4,500 synthetic)
  • Test Set: 3,000 samples (3,000 real - 1,000 per language)

Data Composition

Training Set (18,000 rows)

• Real Data: 7,500 samples
  • Cebuano: 2,500
  • Ilocano: 2,500
  • Tagalog: 2,500
• Synthetic Data: 10,500 samples
  • Synthetic Anchor (from curated sources): 3,298 samples
  • Synthetic Not-Anchor: 7,202 samples
  • Distribution: 3,500 per language

Validation Set (9,000 rows)

• Real Data: 4,500 samples
  • Cebuano: 1,500
  • Ilocano: 1,500
  • Tagalog: 1,500
• Synthetic Data: 4,500 samples
  • Only from synthetic not-anchor sources (no anchor data)
  • Distribution: 1,500 per language

Test Set (3,000 rows)

• All Real Data: 3,000 samples
  • Cebuano: 1,000
  • Ilocano: 1,000
  • Tagalog: 1,000

Data Sources

1. Real Dataset Folder

   • Cebuano: cebuano_sharegpt_5k_random_real.jsonl (5,000 samples)
   • Ilocano: ilocano_v2_sharegpt_5k_random.jsonl (5,000 samples)
   • Tagalog: Tagalog_real_and_augmented_modified.jsonl (5,241 samples)

2. Real and Augmented Tagalog Folder

   • Real: filipino_aya_sharegpt_tagalog_real.jsonl (1,241 samples)
   • Synthetic: filipino_augmented_sharegpt_for_training.jsonl (4,000 samples)

3. Synthetic Anchors Folder

   • Cebuano: cebuano_sharegpt_curated.jsonl (1,098 samples)
   • Ilocano: ilocano_sharegpt_for_training.jsonl (1,100 samples)
   • Tagalog: filipino_curated_sharegpt_for_training_modified.jsonl (1,100 samples)

4. Synthetic Not Anchor Folder

   • All datasets: all_datasets_changed_sharegpt_shuffled.jsonl (12,015 samples)

Data Splitting Strategy

The dataset was carefully split with the following constraints:

• Test Set: Balanced across all 3 languages with only real data (1,000 per language)
• Validation Set: Balanced across languages with synthetic data from non-anchor sources only (no synthetic anchor data)
• Training Set: Includes ALL synthetic anchor data to maximize training capacity while maintaining language balance

This strategy ensures:

• Fair evaluation across all languages
• Validation on both real and synthetic data (but not anchor data)
• Maximum use of high-quality synthetic anchor data for training

Model Performance

Evaluation Metrics

• Test Set (3,000 real samples):
  • Balanced evaluation across Cebuano, Ilocano, Tagalog

Supported Tasks

• Instruction following
• Question answering
• Conversational AI
• Text generation in Philippine languages

Usage

Installation

pip install transformers torch

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "nielle003/Gemma_3_4B_cebuano_ilocano_tagalog"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Example input in Tagalog
prompt = "Ano ang pinakamahusay na paraan para matuto ng programming?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=128)
response = tokenizer.decode(outputs[0])
print(response)

With Hugging Face Pipeline

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="nielle003/Gemma_3_4B_cebuano_ilocano_tagalog",
    device=0  # 0 for GPU, -1 for CPU
)

result = generator("Paano gumawa ng masarap na adobo?", max_length=150)
print(result[0]['generated_text'])

Technical Details

Model Architecture

• Base: Google Gemma 3 4B
• Parameters: 4 billion
• Context Length: 8,192 tokens
• Precision: bfloat16 (recommended), float16, float32

Training Configuration

• Framework: Hugging Face Transformers
• Training Approach: Supervised Fine-tuning (SFT)
• Data Format: JSONL (JSON Lines)

Limitations

• Model is optimized for Cebuano, Ilocano, and Tagalog
• May have limited performance on other languages
• Generated content should be reviewed for accuracy
• Not suitable for production use without additional validation

License

This model is licensed under the MIT License. See the LICENSE file for details.

Citation

If you use this model, please cite:

@misc{gemma_3_4b_philippine_languages,
  author = "Barcelona, Nielle E. and Crisologo Aaron",
  title = {A Lightweight SLM-based Specific Question Answer in Cebuano and ilocano for Edge Devices},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/nielle003/Gemma_3_4B_cebuano_ilocano_tagalog}}
}

Dataset Attribution

• Cebuano Data: ShareGPT format datasets
• Ilocano Data: ShareGPT format datasets
• Tagalog Data: ShareGPT format datasets + Aya dataset

Acknowledgments

• Google for the Gemma model
• Hugging Face for the model hosting platform
• All data contributors and annotators

Contact

For questions or issues, please reach out through Hugging Face Model Hub.

Last Updated: March 2024 Model Version: 1.0