- Elixir 1.19+ / Erlang OTP 27+ —
brew install elixiron macOS. - Postgres 14+ with the
pgvectorextension.brew install postgresql@18 pgvector brew services start postgresql@18
- An OpenAI-compatible or Anthropic-compatible LLM endpoint. Three
required env vars wire the chat client up — see "First boot" below
for worked examples.
LLM_API_KEY— bearer token sent asAuthorization: Bearer ….LLM_API_BASE— base URL the provider request hangs off (e.g.https://api.openai.com/v1,https://api.kimi.com/coding/v1,https://api.deepseek.com/v1,https://api.minimax.io/anthropic,http://localhost:11434/v1for Ollama, …).LLM_MODEL— model id sent in the request body (e.g.gpt-4o-mini,kimi-for-coding,deepseek-chat,MiniMax-M3).- (optional)
LLM_API_FORMAT—openai(default) calls/chat/completions;anthropiccalls/v1/messagesunderLLM_API_BASEand uses Anthropic content/tool/thinking blocks. - (optional)
LLM_USER_AGENT(defaultneoharness/0.1). - (optional)
LLM_THINKING_STYLE—anthropicto emit{"thinking": {"type": "enabled" | "disabled"}}(Kimi, DeepSeek reasoning),openaito emitreasoning_effort: "medium" | "minimal"(gpt-5 / o-series), unset/noneto drop the field. This only affectsLLM_API_FORMAT=openai; the Anthropic adapter maps harness thinking to{"thinking": {"type": "adaptive" | "disabled"}}. - (optional)
LLM_MAX_TOKENS— response-token cap for both API formats. Anthropic-compatible APIs require a value and default to4096when omitted; OpenAI-compatible APIs omit the field when unset and use the provider default. Set8192or16384when your model supports longer visible replies. - (optional)
LLM_PROVIDER_LABEL— string surfaced asllm.providerin OpenTelemetry spans (defaultopenai-compat).
- (optional)
OPENAI_API_KEYfor (a) semantic memory search withtext-embedding-3-small, (b) voice-note transcription via Whisper (whisper-1by default; override withOPENAI_WHISPER_MODEL), and (c) OpenAI-backedimage_generate/image_edit. Without the key, memory falls back to Postgres full-text search, inbound voice notes are surfaced to the model as[voice message — transcription unavailable]markers, and OpenAI image tools hide themselves. - (optional)
MINIMAX_API_KEYfor MiniMax media tools:image_generate provider=minimax,voice_generate, andmusic_generate. If unset andLLM_API_BASEalready points at MiniMax, these tools reuseLLM_API_KEY. - (optional)
MAPTILER_API_KEYfor/atlasmap tiles (client-side MapLibre rendering) and theplace_lookuptool. Atlas extracts place names from persona memories and geocodes them with MapTiler when you click "Refresh from memories";place_lookupgeocodes a single query and renders a web chat place card. Without a key,/atlasfalls back to the decorative SVG. Korea-first defaults areATLAS_GEOCODING_COUNTRY=krandATLAS_GEOCODING_LANGUAGE=ko,en; setATLAS_GEOCODING_PROXIMITY=<lon>,<lat>to bias ambiguous searches around a home/work area. OptionalATLAS_MAP_TILES_KEY(a separate, domain-restricted browser key for quota isolation — falls back toMAPTILER_API_KEY) andATLAS_MAP_STYLE(defaultdataviz-dark) override the tiles key and map style respectively. - (optional)
TELEGRAM_BOT_TOKEN,DISCORD_BOT_TOKENfor chat connectors. Unset → connectors silently skip boot. - (optional)
SCHEDULE_TIMEZONE— IANA zone (e.g.America/Los_Angeles) used to interpret every cron expression inschedules.jsonc. DefaultEtc/UTC. Restart to apply. - The built-in Web connector (the Phoenix LiveView chat) is always
on — no external service, no supervisor child. To disable it,
config :neoharness, Neoharness.Connectors.Web, enabled: falseinconfig/config.exs;ask_userwill then skip the:webbranch when deciding where to deliver questions.
cp .env.example .env
# edit .env — LLM_API_KEY, LLM_API_BASE, LLM_MODEL are required
mix deps.get
mix ecto.setup # creates the db and runs every migration
iex -S mixconfig/runtime.exs auto-loads .env from the project root at boot
via Dotenvy — no source .env
needed. Anything already exported in your shell wins, so prod / CI
env vars override the file. Dotenvy handles the usual dialect
(export prefix, quoted values, # comments, multiline values).
Worked env-var blocks:
# OpenAI
export LLM_API_KEY=sk-...
export LLM_API_BASE=https://api.openai.com/v1
export LLM_API_FORMAT=openai
export LLM_MODEL=gpt-4o-mini
export LLM_THINKING_STYLE=openai
export LLM_PROVIDER_LABEL=openai
# MiniMax M3 (Anthropic-compatible, recommended for interleaved thinking/tool use)
export LLM_API_KEY=...
export LLM_API_BASE=https://api.minimax.io/anthropic
export LLM_API_FORMAT=anthropic
export LLM_MODEL=MiniMax-M3
export LLM_MAX_TOKENS=8192
export LLM_CONTEXT_CUTOFF_TOKENS=800000
export LLM_PROVIDER_LABEL=minimax-m3
# Optional MiniMax media generation:
export IMAGE_GENERATION_PROVIDER=minimax
export MINIMAX_API_KEY=... # can match LLM_API_KEY
export MINIMAX_VOICE_ID=...
# Optional Memory Atlas geocoding and map tiles:
export MAPTILER_API_KEY=...
export ATLAS_GEOCODING_COUNTRY=kr
export ATLAS_GEOCODING_LANGUAGE=ko,en
export ATLAS_GEOCODING_PROXIMITY=127.0276,37.4979
# Optional Atlas map (browser tiles); falls back to MAPTILER_API_KEY:
export ATLAS_MAP_TILES_KEY=your-domain-restricted-key
export ATLAS_MAP_STYLE=dataviz-dark
# Kimi for Coding
export LLM_API_KEY=...
export LLM_API_BASE=https://api.kimi.com/coding/v1
export LLM_API_FORMAT=openai
export LLM_MODEL=kimi-for-coding
export LLM_USER_AGENT=claude-code/2.1.118
export LLM_THINKING_STYLE=anthropic
export LLM_MAX_TOKENS=8192
export LLM_PROVIDER_LABEL=kimi-for-codingOn a healthy boot you'll see something like:
MCP: 0 servers, 0 tools
skills: loaded 0 from /Users/you/.neoharness/skills
Running NeoharnessWeb.Endpoint with cowboy 2.14.2 at 127.0.0.1:4000
From here:
- Open http://localhost:4000 for the web UI.
- From iex:
Neoharness.say("hello")talks to the defaultmainpersona. - http://localhost:4000/docs — these guides rendered as a browsable site (the markdown files are bundled into the app at compile time, so the same URL works from a release).
- http://localhost:4000/dashboard —
Phoenix LiveDashboard. Tabs include Home, Metrics, Processes, Ports,
OS Data (via
:os_mon— CPU, memory, disk), Request Logger (enable a per-session log with a URL param), Ecto Stats (viaecto_psql_extras— slow queries, index usage, cache hit rate). - http://localhost:4000/oban — Oban Web for the job queue (heartbeats, schedules).
- http://localhost:4000/errors — ErrorTracker. Every captured exception (web requests, Oban jobs, and agent/subagent task crashes) grouped with occurrence counts and stacktraces, stored in Postgres.
Inbound images, video, voice notes, and files land under
priv/uploads/<conv_id>/<sha256>.<ext> (gitignored) and are served
back to the web UI via Plug.Static at /uploads/. Single-host
storage — fine for local dev, swap for object storage if you go
multi-host.
The web composer has a paperclip button (file picker) and a
microphone button driven by MediaRecorder in the browser. Telegram and
Discord ingest media inbound automatically (photo / voice / video /
video_note / document / animation on Telegram; any attachment on
Discord). Outbound: the agent uses send_image / send_file tools
to attach media to its replies — see docs/tools.md.
Generated media follows the same path. image_generate can use either
OpenAI or MiniMax and returns a send_image-ready path. MiniMax
voice_generate and music_generate return audio paths that should be
delivered with send_file.
If your model does not support images or videos, set the matching env
var to false in .env:
LLM_SUPPORTS_IMAGE=false # rejects inbound images, strips them from history
LLM_SUPPORTS_VIDEO=false # rejects inbound videos, strips them from historyBoth default to true. When a flag is false:
- Fresh uploads of that media type are rejected immediately with a user-visible message (flash on web, reply on Telegram/Discord).
- Old messages in history are rebuilt as text markers (
[image: name]) instead of multimodal blocks, so the LLM never sees content it can't handle. - If you leave a flag as
truebut the model still rejects the media, the turn surfaces a normal error — it's your responsibility to match the flags to the model's actual capabilities.
Audio and files are always allowed; they are downgraded to text markers automatically.
Most providers accept inline base64 for both images and videos in
image_url / video_url content blocks. Moonshot/Kimi is the
exception: it rejects inline base64 video with invalid part type: video_url and requires the asset to be uploaded via its Files API
first, then referenced by ms://<file_id> in the chat message. Set:
LLM_FILES_API=moonshotWhen this is set, every user turn that carries a video attachment is
pre-uploaded to POST <LLM_API_BASE>/files (purpose=video), the
returned file_id is cached in an ETS table by sha256 for the BEAM
lifetime so re-sends of the same video skip the round-trip, and
Message.to_api/1 emits the ms://<id> reference instead of base64.
Upload failures fall back to inline base64 (which Kimi will then
reject — but the failure is logged and surfaced).
Leave LLM_FILES_API unset for OpenAI / DeepSeek / MiniMax M3's
Anthropic path / providers that handle inline base64 — videos go
straight into the request body. MiniMax M3's Anthropic adapter emits
image / video blocks with base64 source objects.
Neoharness reads user-owned config from ~/.neoharness/:
~/.neoharness/
├── skills/ # skills shared by every persona (optional)
├── mcp.json # MCP servers (optional)
└── agents/
└── <persona>/ # one directory per agent — at least one must exist
├── SOUL.md # personality
├── schedules.jsonc # heartbeats / cron (optional)
├── connectors.jsonc # which telegram chats / discord channels this persona listens on
└── skills/ # persona-scoped skills (optional)
Every agent is a persona at agents/<name>/ — there is no top-level
SOUL.md/schedules.jsonc fallback. On first boot, neoharness
auto-creates agents/main/ with placeholder SOUL.md, an empty
connectors.jsonc, and a schedules.jsonc containing a 30-minute
heartbeat (tick). Edit any of those files; the next
turn (or next heartbeat tick) sees the changes.
Optional starter files for the shared root:
cp examples/mcp.json ~/.neoharness/# Each persona's agent is supervised — kill it and the supervisor restarts it.
:sys.terminate(Process.whereis(Neoharness.Agent.Manager.via("main:main")), :kaboom)
Neoharness.say("still alive?") # worksNeoharness is single-tenant by design: one BEAM process, one Postgres, one user. Deployment means one VPS per user — friends who want to try it stand up their own.
The shipped path is mix release on a small Debian/Ubuntu VPS
joined to a Tailscale tailnet. No public DNS, no public ports, no
Let's Encrypt, no in-app auth. The tailnet is the perimeter: only
people you've invited can reach the box at all, and traffic is
WireGuard-encrypted end-to-end. For sharing with a friend, you invite
them to the tailnet and hand them a http://<host>.<tailnet>.ts.net
URL.
The release runs as your VPS user, so shell_exec and the rest of
the host-touching tooling work as designed — the agent has a real
Linux environment to operate in.
Sample artifact: rel/neoharness.service.example (systemd unit).
A 1 vCPU / 2 GB box is plenty.
1. Provision the OS dependencies.
# Postgres 18 with pgvector. apt's `postgresql` typically lags; use
# the postgres.org apt repo if you want pg18 specifically.
sudo apt-get install -y postgresql postgresql-18-pgvector
sudo -u postgres createuser neoharness --pwprompt
sudo -u postgres createdb -O neoharness neoharness_prod
# Erlang/Elixir to build the release. Use asdf / mise / distro
# packages — anything that gives you Elixir 1.19+ / OTP 27+.
sudo apt-get install -y build-essential git curl libssl-dev libncurses5-dev
# Tailscale.
curl -fsSL https://tailscale.com/install.sh | sh
# Application user.
sudo useradd --create-home --shell /bin/bash neoharness2. Join the tailnet.
sudo tailscale up --hostname=neoharness --sshAuthorize the device in the Tailscale admin console, then note the
MagicDNS name (neoharness.<your-tailnet>.ts.net). Confirm:
tailscale ip -4
tailscale status3. Build the release on the VPS.
sudo -iu neoharness
git clone https://github.com/<you>/neoharness.git ~/build
cd ~/build
mix deps.get --only prod
MIX_ENV=prod mix release
mkdir -p /opt/neoharness/current
cp -a _build/prod/rel/neoharness/. /opt/neoharness/current/Re-run those last four lines + systemctl restart neoharness to
upgrade. There's no hot-code-load story; restarts are cheap.
4. Write /opt/neoharness/.env.
LLM_API_KEY=...
LLM_API_BASE=https://api.openai.com/v1
LLM_MODEL=gpt-4o-mini
DATABASE_URL=ecto://neoharness:<password>@localhost/neoharness_prod
SECRET_KEY_BASE=<mix phx.gen.secret 64>
SIGNING_SALT=<mix phx.gen.secret 32>
# MagicDNS name from `tailscale status`. Used for absolute URLs the
# LiveView generates — must match what friends type in.
PHX_HOST=neoharness.<your-tailnet>.ts.net
PORT=4000sudo chown neoharness:neoharness /opt/neoharness/.env
sudo chmod 600 /opt/neoharness/.env5. Lock the firewall down to the tailnet.
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow in on tailscale0
sudo ufw allow in on tailscale0 to any port 4000
sudo ufw allow 22/tcp # drop this if you only SSH via `tailscale ssh`
sudo ufw enableThe endpoint binds 0.0.0.0:4000, but the only interface that can
reach it is tailscale0. The public IP answers nothing on 4000.
6. Wire up systemd.
sudo cp /opt/neoharness/current/rel/neoharness.service.example \
/etc/systemd/system/neoharness.service
sudo systemctl daemon-reload
sudo systemctl enable --now neoharness
journalctl -u neoharness -f # expect "Running NeoharnessWeb.Endpoint"Neoharness.Release.migrate/0 runs at the top of bin/start, so the
init migration applies automatically on first boot (and any future
ones on upgrade).
7. Visit it.
From any device on your tailnet:
http://neoharness.<your-tailnet>.ts.net:4000. LiveDashboard at
/dashboard and Oban Web at /oban are reachable on the same URL.
On boot, neoharness checks that the release can load core Elixir
protocol implementation modules such as Collectable.BitString and
Inspect.PID. If startup fails with missing required Elixir protocol implementation modules, rebuild the release and recopy the complete
_build/prod/rel/neoharness/ directory. A partial or mismatched release
copy can otherwise surface later as noisy FORMATTER CRASH logs from
System.cmd/3 or Logger instead of a clear startup failure.
In the Tailscale admin console, invite the friend's email to your tailnet (or share the node directly via "Share node"). They install Tailscale on their device, accept the invite, and the MagicDNS URL just resolves for them. Revoking access is the inverse: remove them from the tailnet, the URL stops resolving.
There's still no in-app authorization, so anyone with tailnet access to the box drives the same agent — they see your conversations, can trigger your tools, etc. If you want isolated state per friend, give each friend their own VPS; the codebase is single-tenant.
State that survives a rebuild lives in two places:
- The Postgres database (messages, memories, conversations, schedules, Oban jobs).
/home/neoharness/.neoharness/(per-persona SOUL.md, schedules.jsonc, per-persona skills, MCP config).
A nightly pg_dump of neoharness_prod plus a tar of
~/.neoharness to an off-box destination is the minimum survivor
set. priv/uploads/ is recoverable from message history if you don't
back it up; back it up if you care about exact bytes.