Situation
The recreational vehicle (RV) market lacks an efficient, AI-powered question-answering system to provide quick and accurate information about various RV models.
Task
Develop a specialized AI model capable of answering detailed questions about different RV models, including specifications, features, and pricing.
Action
- Prepared a dataset of RV-specific questions and answers.
- Fine-tuned the LLaMA 3.1 70B Instruct model using the Unsloth library for efficient training.
- Implemented LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.
- Optimized the training process for speed and memory efficiency.
Training Configuration
- Model: LLaMA 3.1 70B Instruct
- Library: Unsloth
- Fine-tuning Method: LoRA
- Dataset Size: 16 examples
- Batch Size: 2
- Gradient Accumulation Steps: 4
- Learning Rate: 2e-4
- Training Steps: 60
Result
The fine-tuned model demonstrates impressive capabilities in answering RV-specific questions:
Sample Q&A
Q: Does the Jayco Eagle have a queen size bed?
A: Yes, the Jayco Eagle has 1 queen size bed.
Q: What is the MSRP of the Keystone Cougar?
A: The MSRP of the Keystone Cougar is $28,603.
Performance Metrics
- Training Time: 6.0 minutes
- Peak Memory Usage: 40.941 GB
- Memory Usage for Training: 2.082 GB (2.63% of available GPU memory)
The model has been successfully uploaded to Hugging Face Hub and is available for further use and experimentation.