ModelVault is hiring a
Remote Senior Software Engineer
ModelVault is building the world’s first fully-integrated, on-prem AI appliance — a plug-and-play AI server preloaded with software, models, and tools that let businesses (and eventually consumers) run powerful AI models locally, privately, and securely. We're looking for a Senior Software Engineer to help us build the core developer experience and control plane for our on-prem systems — everything from containerized runtime orchestration to local APIs, CLI tools, OTA update flows, and lightweight UIs. You'll be one of the first engineers on the team and play a foundational role in shaping our system architecture, tooling, and technical culture.
🚀 What You’ll Do- Design and build the software layer that makes ModelVault “just work” — including:
- Local APIs and system services
- Developer-facing CLI tools
- Simple admin web UI or dashboards
- OTA update infrastructure
- Wrap system-level functionality into intuitive interfaces for local users (e.g., “start model,” “view logs,” “update software”)
- Integrate and manage containerized LLMs and vision models (e.g. Llama 3, Mistral, SDXL) for local inference
- Collaborate with our systems and hardware engineers to optimize performance across the full stack — from GPU runtimes to dev UX
- Participate in architecture decisions related to system orchestration, hardware/software coordination, and secure update delivery
- Interface directly with early users to gather feedback, debug edge cases, and iterate quickly
- Contribute to hiring and scaling the engineering team over the next 12 months.
🧠 You Might Be a Fit If You- Have 5+ years of software engineering experience — ideally in devtools, infra, or platform teams
- Are comfortable working close to the system — building tools that wrap around Docker, shell scripts, or system services
- Enjoy working across the stack — from lightweight frontend dashboards to backend APIs to scripting hardware operations
- Have experience building interfaces for technical users (CLIs, admin panels, config systems)
- Have worked with containerized environments and understand Docker, Compose, systemd, etc.
- Like solving ambiguous technical problems with minimal abstraction — and high leverage
⭐️ Bonus Points- Built tooling or platforms for AI workloads (e.g. inference runners, LLM wrappers, logging for local models)
- Experience with GPU systems, Nvidia drivers, CUDA, or model performance tuning
- Familiarity with update mechanisms (auto-updaters, patch delivery, rollback safety)
- Past experience with embedded systems, local-first products, or edge computing
⚙️ Our Tech Stack (Currently)- Python, FastAPI, Shell scripting, Docker / Compose
- Click or Typer for CLI
- React / Next.js (if building dashboards)
- OS: Ubuntu
- LLM tooling: llama.cpp, vLLM, Ollama, GGUF, HuggingFace
- Remote: Tailscale, SSH, or light VPN-based access
🌍 Why Join ModelVault?- Help define a new category: secure, local AI for everyone — starting with businesses, expanding to the world
- Work on real, shipped hardware + software that users will touch
- Be part of the founding engineering team with high autonomy and deep ownership
- Remote-first, async-friendly culture with deep focus and high velocity
Please mention the word COMPREHENSIVE when applying to show you read the job post completely (#RMjE2LjczLjIxNi4xNDY=). This is a feature to avoid fake spam applicants. Companies can search these words to find applicants that read this and instantly see they're human.
Salary and compensation
$60,000 — $80,000/year
Benefits
🌎 Distributed team
⏰ Async
🏖 Paid time off
📚 Learning budget
💰 Profit sharing
🚫 No politics at work
How do you apply?
Please submit your application using the link below. We aim to respond to you within one week.
👉 Please reference you found the job on Remote OK, this helps us get more companies to post here, thanks!
When applying for jobs, you should NEVER have to pay to apply. You should also NEVER have to pay to buy equipment which they then pay you back for later. Also never pay for trainings you have to do. Those are scams! NEVER PAY FOR ANYTHING! Posts that link to pages with "how to work online" are also scams. Don't use them or pay for them. Also always verify you're actually talking to the company in the job post and not an imposter. A good idea is to check the domain name for the site/email and see if it's the actual company's main domain name. Scams in remote work are rampant, be careful! Read more to avoid scams. When clicking on the button to apply above, you will leave Remote OK and go to the job application page for that company outside this site. Remote OK accepts no liability or responsibility as a consequence of any reliance upon information on there (external sites) or here.
|
Frontpage
🏝 Remote jobs
🌗 Dark mode
👩💻 Hire remote workers
🚨 Post a remote job
🍱 Compact mode
✍️ Remote work blog new
Health insurance for teams
Hacker News mode
Safe for work mode
