Note: The job is a remote job and is open to candidates in USA. Steneral Consulting is seeking an AI/ML Engineer to design and implement secure, scalable AI solutions. The role involves optimizing AI models for local infrastructure and ensuring compliance with regulatory requirements.
Responsibilities
- Design, deploy, and optimize open-source LLMs and AI frameworks for on-premises, bare-metal/private hardware clusters
- Build, maintain, and secure local AI inference pipelines, advanced fine-tuning/embedding workflows, and RAG (Retrieval-Augmented Generation) systems
- Optimize AI models and workflows for maximum efficiency (latency, throughput, CPU/GPU/memory usage) on bare-metal infrastructure
- Ensure complete data isolation, integrating AI with internal data sources while adhering to compliance and regulatory requirements
- Evaluate and select open-source, local-first AI tools (vector databases, orchestration frameworks, model serving layers)
- Collaborate with engineering and compliance teams to align AI solutions with regulatory needs
- May perform additional duties as assigned
Skills
- Extensive local (non-cloud) AI deployment experience, especially with open-source LLMs (e.g., LLaMA, Mistral) in air-gapped/private environments
- Deep expertise in AI frameworks like PyTorch, Hugging Face, LangChain, LlamaIndex
- Strong experience with RAG architectures, embeddings, semantic search, and vector databases (e.g., FAISS, Qdrant, Milvus, Chroma)
- Proficient in containerizing AI workloads (Docker/Kubernetes) and managing GPU-based compute environments
- Solid understanding of advanced ML concepts (e.g., LoRA/QLoRA fine-tuning, prompt engineering, model quantization formats like GGUF, AWQ, EXL2)
- Ability to work autonomously in isolated, non-cloud development environments
- Experience in compliance-driven or regulated industries (e.g., fintech, legal-tech)
- Familiarity with local-first agentic workflows, Model Context Protocol, and developing internal developer copilots
Company Overview