ASTRO
A voice assistant that lives inside Ableton Live. It understands what you ask and carries it out through checked actions.
- By voice
- you ask Ableton Live to do something, and it does it
- Almost 200 actions
- the assistant can carry out inside the software
- Hands on the instrument
- while you play or record, without reaching for the mouse

The problem
People who make music on their own do many jobs: they compose, arrange, mix and play live. In Ableton Live, one of the most widely used programs for producing music, every move goes through menus, shortcuts and windows. When your hands are on the instrument, stopping to look for a command breaks the flow.
What it does
ASTRO is an assistant that lives inside Ableton Live. You ask it for something out loud or in writing, even from your phone, and it does it: it works on tracks, instruments, the mixer, clips, scenes and automation, and knows almost 200 different actions. A discreet panel on the Mac shows what is happening and then disappears.
It grew out of the way I work in the studio and on stage, for Ableton Live 11 and 12 on macOS.
The choices that matter
- It talks to the software from the inside. Instead of faking mouse and keyboard, which is fragile, it uses the channel Ableton provides for being controlled from within.
- Rules first, then AI. Simple commands are recognised and carried out straight away, without AI. AI comes in only when the request is more complex.
- AI proposes, the software executes. The AI never touches Ableton directly. It drafts a plan made only of planned actions, waits for confirmation, and every action is checked before and after. You always know what it can do and what it has done.
- When something goes wrong, it says so. If the AI service fails several times in a row, the assistant pauses it for a minute and says so clearly, instead of pretending all is well.
Problems solved along the way
- Google blocked the addresses of the server hosting ASTRO, and the AI stopped until I routed the requests through different infrastructure.
- Measuring real use. In June, automated tests and real use were mixed together in the statistics, and the success rate swung between 79% and 21% within a few days. Since then the two have been measured separately.
- Not everything can be driven from the inside. Of the 193 catalogued actions, 110 go through the direct channel. The others need shortcuts or sequences of steps, which are more fragile, and a review found 41 still to be verified.
Where it stands
ASTRO is a private alpha for macOS. My assessment in July was “about 80% functional, not yet ready to sell”. Going further needs a signed app, an Apple Developer account and a beta period. The latest features date from July 2026; since then I have focused on other products.
What I learned
- A closed list of actions makes AI explainable. You always know what it can do and what it has done.
- Make it safe before you simplify. Before removing the old parts I added automated tests, and that surfaced two hidden bugs in the metronome.
- Automated tests and real use must be measured separately.
- Check before you rebuild. What you need often already exists and is more complete than it looks.
Try it
There is no video yet. The page for musicians is at silvestronelcosmo.com/astro, and I can show ASTRO live on a call.


How the information moves
The method I use with clients, applied to this project: follow the information from where it starts to how you know everything works.
- Where it startsAn Ableton Live session with tracks, instruments and clips, and the musician at the Mac or with a phone in hand.
- How it arrivesA spoken or typed request. Meanwhile the software keeps the assistant up to date on the state of the session.
- How it is describedEvery request becomes a clear intention, picked from a closed list of actions, each with its own rules.
- What happens to itSimple commands are recognised straight away; for complex ones the AI drafts a plan made only of actions from the list.
- Where it livesThe Mac remembers the recent context; accounts and statistics live in an online service.
- Who decidesThe musician confirms the plan, and every action is checked and must be one of the permitted ones.
- What happens nextThe assistant carries out the actions inside Ableton Live, from within the software whenever possible.
- How we know it worksEvery action reports how it went, and a before-and-after check flags when it had no effect.
For those who want the technical details
How it is built, for people who work in software or want to know what is underneath. Every number has a source.
- A Python service on the Mac receives commands, spoken, typed or from the phone, and forwards them over TCP, using the NDJSON astro/v1 protocol, to a Remote Script inside Live.
- Rule-based recognition in 22 modules in priority order. Only when no rule matches does Gemini 2.5 Flash step in, through the backend.
- The Mentor proposes a plan with its tools switched off. After confirmation, only skills with a validated Pydantic schema run (130, of which 13 high-risk), chosen from a server-side allowlist.
- FastAPI backend on Render with Supabase authentication and Stripe payments; model calls through a Cloudflare Worker; optional local mode with faster-whisper, Ollama and Piper.
- iOS app with Capacitor, desktop panel in PySide6, interface in six languages. Contracts, design tokens and translations come from a single source, guarded by tests.
- Built with several AI coding agents taking turns (Claude, Gemini and Codex), coordinated through a handover document with shared rules.
- Actions available to the AI
- 130 skills with a validated schemasource: contratto astro_v2_skills.json
- Catalogued Live actions
- 193, of which 110 via the native scriptsource: action_kb.json, 15/07/2026
- Automated tests
- 570 in the core and 258 in the backendsource: HANDOFF.md, 10/07/2026
- Commits
- 428 in the monorepo and 314 in the backendsource: git log, maggio–settembre 2026
- Status
- Private alpha on macOSsource: silvestronelcosmo.com/astro
Tools used: Python, PySide6, Ableton Live Remote Script, NDJSON su TCP, FastAPI, Pydantic, Supabase, Stripe, Gemini, Cloudflare Workers, TypeScript, Capacitor, PyInstaller