AI-POWERED AUTOMATION WITH AZURE

From Prompt to Productivity

A build walkthrough — how we took the way we sort email by hand and turned it into an Azure service that does it for us.
The goal: clear a 5,000-email backlog and keep it clear — no app, no rules engine, no manual sorting.
Hardip Patel · hardip.me

Who I am

  • Hardip Patel — Engineer at Atyantik Technologies.
  • I build full-stack apps and data pipelines for a living.
  • And I over-engineer my own life, one sync pipeline at a time — my music, reading, and health metrics are all auto-tracked.
  • This talk is one of those projects: I pointed it at my inbox.
hardip.me · github.com/knightkill

First: how we sort email by hand

The loop every one of us runs, all day:

  • Open an email — read the sender and subject.
  • Decide — does this actually need me? (a real person, money, my clients…)
  • Act — keep it in Primary, or label it and archive it.
  • Repeat — a few thousand times.
Result: 5,000 unread, the important stuff buried, and eventually you just give up.

Then: the same loop, automated

You, by hand

  • open & read
  • decide: keep / archive
  • label & move
  • repeat ×1000s

The automation

  • Timer fetches new mail
  • Azure OpenAI decides
  • code labels & archives
  • every 10 min, unattended
The trick: write your judgment down once, in plain English, and let the machine apply it to every email.

The build, in four stages

1 · Setup
Azure + Gmail OAuth, once
→
2 · Local
policy + classifier on your machine, dry-run
→
3 · Azure
a timer runs it unattended, every 10 min
→
4 · Attachments
Document Intelligence — homework / PR
Build & test it locally first, then ship the same pipeline to Azure. Attachments are the open piece.

How it decides — without hallucinating

The model must fill a typed verdict — so it can't invent a label.
class TriageVerdict(BaseModel):       # the contract the model must satisfy
    reason: str                       # decide out loud first
    category: Category                # a closed enum, not free text
    labels: list[Label]              # only real Gmail labels
    importance: int = Field(ge=0, le=100)
    keep_in_primary: bool

verdict = client.beta.chat.completions.parse(
    model="gpt-5.4-nano",
    messages=[{"role": "system", "content": policy},   # policy.md = the rules
              {"role": "user",   "content": email}],
    response_format=TriageVerdict,         # ← it can't reply with anything else
).choices[0].message.parsed
Editing policy.md changes behaviour — no code, no redeploy of logic.

How it acts on a real inbox — safely

Guards + dry-run wrap every mutation. Archive = remove the INBOX label.
def apply_verdict(svc, msg, verdict, *, dry_run=True):
    if is_protected(msg):                  # starred / VIP / threads I replied to
        return {"action": "skipped"}
    add    = [label_id(l) for l in verdict.labels]
    remove = [] if verdict.keep_in_primary else ["INBOX"]   # archive out of Primary
    if dry_run:                            # default: log, touch nothing
        return {"action": "dry-run", "add": add, "remove": remove}
    svc.users().messages().modify(
        userId="me", id=msg["id"],
        body={"addLabelIds": add, "removeLabelIds": remove},
    ).execute()                            # idempotent + written to an audit log

Same pipeline, two homes

Local — run_local.py

you run it → policy.md → Azure OpenAI → Gmail → label / archive
  • secrets from .env
  • auth: token.json (browser once)
  • state: local files · dry-run default

Azure — Function

Timer (10 min) → policy.md → Azure OpenAI → Gmail → label / archive
  • secrets from Key Vault (managed identity)
  • auth: token from Key Vault (headless)
  • state: Blob · dry-run via app setting
The middle — policy → classify → act — is the exact same code. Only the trigger, where secrets come from, and the state backend change.

How we set it up — Azure + Gmail

Azure (one-time): create the function with an identity, vault the secrets, deploy.
az functionapp create --runtime python --assign-identity ...   # managed identity
az keyvault secret set --vault-name $KV -n gmail-token-json --file token.json
func azure functionapp publish $APP --python --build remote
Gmail (one-time): consent once, then run headless forever.
  • OAuth Desktop client → consent in the browser → get a refresh token.
  • Store that token in Key Vault → the cloud function signs in with no browser.

The Azure pieces — and why each

  • Azure OpenAI — runs the classifier (gpt-5.4-nano) with structured output.
  • Functions (Consumption + Timer) — runs every 10 min; serverless, pay-per-run.
  • Key Vault + Managed Identity — holds secrets; no keys ever live in code.
  • Blob Storage — processed-set + audit log, so the function stays stateless.
@app.timer_trigger(schedule="0 */10 * * * *")          # the automation: every 10 min
def handle_scheduled_triage(t): run_triage()

SecretClient(KV_URI, DefaultAzureCredential()).get_secret(name)   # identity, not keys

What we achieved

Before
5,000 unread
everything in one pile
After
Primary holds only
what needs me
  • Runs every 10 minutes, unattended — ~$0 on Consumption (App Insights hard-capped).
  • What we learned: gpt-5 needs max_completion_tokens; Linux Consumption won't remote-build a zip (use func publish); structured output makes it trustworthy, dry-run makes it safe.
  • Open · PRs welcome: attachment-aware triage via Azure Document Intelligence — stubbed, plumbing ready (a good first PR).
github.com/knightkill · the pattern works on any inbox

Reproduce it

It's one repo and a setup guide — point it at your own inbox.

  • git clone → fill .env → follow SETUP.md
  • Azure + Gmail · ~30 minutes · ~$0 to run
  • Defaults to dry-run — safe to try on a real inbox
github.com/knightkill/inbox-triage
QR code linking to github.com/knightkill/inbox-triage

SPEAKER: the map for the rest of the talk. Point at where we are as you go. Stage 4 is dashed = not built.

SPEAKER: the key reassurance — you debug locally, then the identical pipeline runs in the cloud. Differences are just the edges.

SPEAKER: numbers drift as the timer runs, so this slide stays qualitative. If you want a hard count, pull it live from the audit log right before the talk.

SPEAKER: The takeaway slide. "Clone it, fill .env, follow SETUP.md — Azure + Gmail in ~30 min." Leave this up during Q&A.