RAGtime: Lessons from Engineering an AI System
RAG looks deceptively simple on a whiteboard: ingest documents, create embeddings, retrieve context, ask a model. Engineering a reliable AI system around that pipeline is much harder.
This talk is a field report from building PrAIvate, a privacy-first open-source RAG platform. It focuses on the conditions that made rapid iteration possible: reproducible infrastructure with Ansible, build pipelines, test and evaluation frameworks, observability, explicit agent configuration for AI-assisted development, and repeated architectural reassessment.
The most useful lessons emerged when local choices stopped being local. Retrieval quality affected chunking and context design. Hardware constraints changed model and reranker decisions. Evaluation exposed weaknesses hidden by plausible-looking answers. Faster AI-assisted implementation made alternatives cheaper to explore, but increased the need for architecture rules, automated tests, and disciplined review.
The result is not a recipe for one RAG stack, but a set of practical lessons for building AI systems that can be changed, measured, reproduced, and operated.
Book Now