Infra CLI

Mirrored from iblai/infra-cli · README.md. This page is generated — edit it in the repository, not here.

Interactive CLI for provisioning and configuring the ibl.ai platform on AWS or GCP (single-server). End-to-end infrastructure with Terraform, full application setup with Ansible. Can also bootstrap existing servers (any cloud or bare metal) without Terraform.

AWS GCP Terraform Python 3.11+ License: MIT

Note: This repository contains the installation and infrastructure provisioning tooling. Access to the ibl.ai Docker images and platform codebase requires a license. To get started, reach out at ibl.ai/contact.

---

Quick start

Five steps from zero to a running platform:

# 1. Install (needs Python 3.11+, Terraform on PATH)
git clone git@github.com:iblai/iblai-infra-ops.git && cd iblai-infra-ops
uv sync                        # AWS only
uv sync --extra gcp            # AWS + GCP support

# 2. Verify your cloud credentials have what's needed
uv run iblai infra permissions --check                                  # AWS
uv run iblai infra permissions --provider gcp --check --project <ID>    # GCP

# 3. Provision the infrastructure (interactive wizard — pick AWS or GCP)
uv run iblai infra provision

# 4. Install the platform on the new VM (Ansible over SSH)
uv run iblai infra setup <project-name>

# 5. Open the app
#    https://learn.<your-domain>

Prefer no prompts? Steps 3–4 also run from a .env file — see Non-interactive.

Deploying on Google Cloud? The flow above is identical; read the short GCP guide first for GCP prerequisites (project, APIs, auth) and cert/DNS behavior.

Prerequisites

AWS credentials are required on every deployment, including bare metal and GCP. They are not for hosting; they cover two things:

• ECR — pulling the platform container images.

• S3 — the platform keeps media, static assets, and backups in three buckets. Provisioning on AWS creates them for you. On any other target, create them yourself before setup, named:

  <project>-<environment>-<domain-with-dots-as-dashes>-backups
  <project>-<environment>-<domain-with-dots-as-dashes>-dm-media
  <project>-<environment>-<domain-with-dots-as-dashes>-dm-static   # public-read

Installed automatically as Python dependencies: ansible-core (runs the setup), boto3 (AWS SDK), and — with --extra gcp — the Google Cloud SDKs. Terraform is called as a subprocess and must be installed separately.

What setup installs on the target server: Docker Engine + compose, pyenv + Python 3.11.8, AWS CLI v2, and iblai-cli-ops (the platform management CLI — private repository; unauthenticated requests see a 404).

Install

curl -LsSf https://astral.sh/uv/install.sh | sh     # install uv if needed

git clone git@github.com:iblai/iblai-infra-ops.git
cd iblai-infra-ops
uv sync --extra gcp                                  # or plain `uv sync` for AWS only

Verify:

uv run iblai --version
terraform --version

Every command below can be run as uv run iblai ..., or just iblai ... after source .venv/bin/activate.

Usage

Run iblai infra with no arguments for an interactive menu of all commands.

1. Check cloud permissions

iblai infra permissions [--check]                                  # AWS: IAM policy JSON + dry-run
iblai infra permissions --provider gcp [--check --project <ID>]    # GCP: roles + APIs + probe

2. Provision infrastructure

iblai infra provision

The wizard walks you through, validating as you go:

1. Cloud — AWS or GCP
2. Credentials — AWS: profile / keys / env vars · GCP: project ID, region/zone, ADC or key file
3. Topology — AWS: single-server, multi-server, or call-server (LiveKit) · GCP: single-server
4. Project & compute — name, environment, machine/instance type, disk (min 100 GB; a 32 GB-RAM pick prints a heads-up — 64 GB recommended when AI features are on)
5. Network & SSH — CIDR, your IP for SSH access (auto-detected), SSH key (generate or provide)
6. Domain & certificates — AWS: Route53 + ACM / upload / none · GCP: Cloud DNS + Google-managed cert (validates asynchronously, 10–60 min) / upload / none
7. WAF (optional, AWS single-server) — AWS WAFv2 on the ALB; skippable, add later with iblai infra waf enable
8. Review — full summary before anything is created

Terraform then runs with live per-resource progress. The results panel shows the VM IP, SSH command, and app URL.

3. Set up the platform

iblai infra setup <name>       # a Terraform-provisioned environment (IP/domain/key auto-filled)
iblai infra setup              # any existing server (bare metal / other cloud) — prompts for everything

Prompts for: release tag of iblai-prod-images (the one version knob — the matching iblai-cli-ops version is resolved automatically from its pin), tenant platform name (blank = default), enable-AI toggle, optional integrations (SMTP / Stripe / Google SSO / Microsoft SSO — each off unless configured), GitHub PAT, AWS credentials (ECR + S3), OpenAI key (optional), and super admin credentials.

Everything optional can wait. SMTP, SSO, Stripe and the OpenAI key are each skippable here — the platform comes up without them. Only the GitHub PAT, AWS credentials and admin credentials are needed on the first run. Add any of them later with iblai infra configure <name> — no need to re-run setup.

Installing from your own repos. Three prompts control where the private packages come from. Press Enter to accept the defaults, or point them at a fork or per-deployment copy:

Both repo prompts also accept a repo/subdir path when the two packages live in one repository. The GitHub PAT needs Contents: Read on whichever repositories you name. The same three values are available as GITHUB_ORG, CLI_OPS_REPO and PROD_IMAGES_REPO in .env.setup.example.

The playbook runs 16 roles in phases:

4. Non-interactive provision + setup (.env file)

Same Terraform + Ansible, zero prompts. Single-server only.

# Provision — AWS
cp .env.provision.example .env && $EDITOR .env
iblai infra provision-env -f .env

# Provision — GCP (sets PROVIDER=gcp)
cp .env.provision.gcp.example .env && $EDITOR .env
iblai infra provision-env -f .env

# Set up the provisioned VM
cp .env.setup.example .env.setup && $EDITOR .env.setup
iblai infra setup-env <project-name> -f .env.setup

Free-standing server (no Terraform): omit the project name and add TARGET_HOST, SSH_PRIVATE_KEY_PATH, BASE_DOMAIN, PROJECT_NAME to .env.setup.

Sample .env files (every key documented inline — copy, edit, run):

Populated .env files are gitignored; secrets are never persisted to state.json — they ride --extra-vars into Ansible at run time only.

5. Re-setup an existing environment

iblai infra resetup <name>

Points a running environment at a new domain with fresh secrets: prompts for the new base domain, prod-images release tag, and credentials; rotates all secrets, syncs DB passwords, restarts services. No Terraform changes.

6. Verify DNS

iblai infra dns check <name>            # check once
iblai infra dns check <name> --watch    # re-check until everything resolves

Unless the deployment manages DNS itself (Route53 + ACM, or an existing Cloud DNS zone), you create the records — see the list printed after provisioning, also saved as dns-records.txt in the workspace. Provisioning offers to run this check straight away, and it can be re-run at any point while you wait for the records to appear.

Per subdomain it reports whether the name resolves and whether it points at this deployment's load balancer — a record resolving to some other host is flagged rather than passed. Certificate state is shown alongside, since an ACM or Google-managed certificate cannot validate until the records exist. Lookups go to public resolvers, so a stale local cache can't mask a failure. Exits non-zero while anything is unresolved.

iblai infra setup <name> runs the same check first and asks before installing against domains that don't resolve; --skip-dns-check bypasses it.

7. Optional feature toggles (post-provision)

Everything optional at setup can be added later against a running environment, without re-running setup:

iblai infra configure <name>       # menu of everything below

The environment must already be provisioned and set up. Only the relevant Ansible role re-runs, and it needs just the SSH key already recorded in the project — no GitHub token or AWS keys. Most changes are live immediately; the two exceptions are noted below.

Email (SMTP)

iblai infra smtp enable <name>               # prompts for host, port, credentials, sender
iblai infra smtp enable-env <name> -f .env   # non-interactive (SMTP_* keys)
iblai infra smtp disable <name> [--yes]
iblai infra smtp status <name>               # reads the live values off the server

⚠️ The only feature needing a restart — the settings reach the services as environment variables, so Data Manager and Open edX are recreated. The command asks first; --no-restart skips it (the change stays inert until they restart), --yes skips the prompt.

Google SSO

iblai infra sso google <name>

Register an OAuth client in Google Cloud Console first, with authorised redirect URI https://learn.<your-domain>/auth/complete/google-oauth2/. Prompts for client ID, secret, and an optional Workspace domain to restrict to. Live immediately.

Microsoft SSO

iblai infra sso microsoft <name> [--yes]

Register an application in Azure first, with redirect URI https://learn.<your-domain>/auth/complete/azuread-oauth2/. Prompts for client ID, secret and tenant ID.

⚠️ Restarts Open edX — it changes settings read only at boot. The command warns first.

Stripe billing

iblai infra stripe enable <name>
iblai infra stripe enable-env <name> -f .env   # non-interactive (STRIPE_* keys)

Prompts for test/live mode, secret and publishable keys, and optional pricing-table and webhook values. Live mode warns that real cards will be charged. Live immediately.

LLM API key

iblai infra llm set-key <name>
iblai infra llm set-key <name> --provider anthropic --api-key <key>   # non-interactive

Sets or rotates the credential the mentor service uses. Supported providers are openai (the default) and anthropic. Setting a key also makes that provider the preferred one, since the platform picks the preferred credential with no tie-break when several are marked. Live immediately.

Tenant platform

iblai infra platform create <name>
iblai infra platform create <name> --platform-name <tenant>

Creates a platform alongside the default main one, via the platform launcher so it comes up fully wired. The tenant admin credentials are printed once and stored nowhere else — save them.

Clone a SPA

iblai infra spa clone <name>                 # prompts for source, name and domain
iblai infra spa list <name>                  # what's deployed, with ports
iblai infra spa remove <name> --spa <clone>

Runs a copy of a deployed SPA alongside the original — its own port, its own domain — so it can be customised without touching the one the platform depends on. The copy starts with the source's running environment file, so it behaves identically until you change it:

# customise, then restart
/ibl/app/ibl-spa/<clone>/.env.<clone>
cd /ibl/app/ibl-spa/<clone> && docker compose up -d

Ports are allocated from 5060 (stock SPAs hold 5000–5009). The nginx block lands in /etc/nginx/conf.d/custom_domains/, which survives ibl global-proxy regeneration. Served over HTTP — point the domain at the server and put it behind whatever already terminates TLS. remove refuses the platform's own SPAs.

WAF (AWS single-server only)

iblai infra waf enable [<name>]               # WAFv2 on the ALB
iblai infra waf enable-env [<name>] -f .env   # non-interactive (WAF_ALLOWED_IPS)
iblai infra waf disable <name> [--yes]
iblai infra waf status [<name>]

The only one of these that changes infrastructure rather than platform config — it re-applies Terraform.

8. Launch from AMI (AWS, CI/CD)

One-shot Terraform + Ansible from a pre-built AMI:

iblai infra launch-env                       # .env-driven (cp .env.example .env first)

iblai infra launch \                         # fully flag-driven
  --ami-id $AMI_ID --domain $DOMAIN --hosted-zone-id $HOSTED_ZONE_ID \
  --aws-key-id $AWS_ACCESS_KEY_ID --aws-secret-key $AWS_SECRET_ACCESS_KEY \
  --ssh-public-key "$SSH_PUBLIC_KEY" --ssh-key $SSH_KEY_PATH \
  --git-token $GIT_TOKEN --vpn-ip $VPN_IP \
  --admin-email $ADMIN_EMAIL --admin-password $ADMIN_PASSWORD

See iblai infra launch --help for all options (instance type, --platform-name, SMTP/Stripe/SSO, --enable-ai, ...).

9. Service update (image updates, CI/CD)

Update container images and restart services — no provisioning, no secret rotation:

iblai infra service-update --host <ip> --ssh-key ~/.ssh/key.pem --git-token $GIT_TOKEN

Or launch a fresh AMI and swap it into an ALB target group — see iblai infra service-update --help.

10. Manage environments

iblai infra list                # all environments (cloud, type, status)
iblai infra status <name>       # details + outputs for one
iblai infra auth                # switch AWS credentials
iblai infra destroy <name>      # tear everything down

Authentication

The CLI never silently auto-detects credentials — you always choose, interactively on first use, and the session is reused across commands.

AWS — pick one in the wizard: profiles from ~/.aws/ (type to filter), environment variables, or manual access keys.

GCP — two options; the wizard asks which:

Option A — your Google account (local use):

gcloud auth application-default login    # note: `gcloud auth login` alone is NOT enough

Option B — service-account key (CI, or if your org blocks the browser flow above):

PROJECT=<your-project-id>
gcloud iam service-accounts create infra-cli --project=$PROJECT
for R in roles/compute.admin roles/dns.admin roles/iam.serviceAccountUser; do
  gcloud projects add-iam-policy-binding $PROJECT \
    --member="serviceAccount:infra-cli@$PROJECT.iam.gserviceaccount.com" --role=$R --condition=None
done
gcloud iam service-accounts keys create ~/infra-cli-key.json \
  --iam-account="infra-cli@$PROJECT.iam.gserviceaccount.com"

Then choose "Service-account key file" in the wizard (or set GCP_CREDENTIALS_FILE=~/infra-cli-key.json in your .env). Keep the key file out of git; delete the service account when done.

What gets created

AWS (Terraform): VPC with 2 public subnets · EC2 (Ubuntu 22.04, encrypted EBS) · Application Load Balancer (TLS 1.2/1.3) · ACM certificates (DNS-validated, auto-renewed) · security groups (SSH restricted to your IP) · 3 S3 buckets (backups, media, static) · Route53 A-records for all subdomains · optional WAFv2.

GCP (Terraform): VPC + regional subnet · Compute Engine VM (Ubuntu 22.04) · global external Application Load Balancer with a static IP · Google-managed SSL certificate (all subdomains, async validation) · firewall rules (SSH restricted to your IP, health-check ranges) · Cloud DNS A-records. Object storage stays on AWS S3 — details in the GCP guide.

Subdomains — each gets a DNS A-record and is covered by the load balancer's TLS certificate. Replace <your-domain> with your base domain:

• learn.<your-domain>
• preview.learn.<your-domain>
• studio.learn.<your-domain>
• apps.learn.<your-domain>
• meilisearch.learn.<your-domain>
• api.<your-domain>
• api.data.<your-domain>
• asgi.data.<your-domain>
• llm.data.<your-domain>
• base.manager.<your-domain>
• auth.<your-domain>
• os.<your-domain>
• lms.<your-domain>
• platform.<your-domain>
• monitor.<your-domain>
• flowise.<your-domain>
• prometheus.<your-domain>

Platform services (Ansible): Open edX (LMS/CMS + MySQL/Redis/MongoDB/Elasticsearch/Forum) · Data Manager (Django/ASGI/Celery + PostgreSQL 16/Redis/Flowise) · Auth/Mentor/Skills SPAs · monitoring (Prometheus/Grafana) · nginx reverse proxy.

Workspace

Terraform state, generated SSH keys, and project configuration live at ~/.iblai-infra/projects/<project-name>/.

Development

uv sync --extra dev --extra gcp
uv run pytest tests/ -v                                          # 749 tests, ~2s
uv run pytest tests/ --cov=iblai_infra --cov-report=term-missing

iblai-infra-ops/
├── src/iblai_infra/
│   ├── cli.py                  # Typer CLI commands
│   ├── app.py                  # Provision wizard orchestrator
│   ├── models.py               # Pydantic models (the wizard ↔ Terraform contract)
│   ├── env_provision.py        # .env → InfraConfig (provision-env, AWS)
│   ├── gcp_env_provision.py    # .env → InfraConfig (provision-env, GCP)
│   ├── env_setup.py            # .env → SetupConfig  (setup-env)
│   ├── prompts/                # Interactive questionary prompts
│   ├── providers/              # aws.py + gcp.py — cloud SDK helpers
│   ├── features/               # Post-provision toggles (waf.py)
│   ├── terraform/              # Runner + templates
│   │   └── templates/
│   │       ├── aws/            # single-server (+ waf.tf), multi-server, call-server
│   │       └── gcp/            # single-server
│   └── ansible/                # Runner + playbooks + 16 roles
├── docs/                       # GCP guide, development notes
├── tests/                      # 749 tests
└── pyproject.toml

License

Released under the MIT License.