How this audit is made — and how it improves

The pipeline

1. Watchlist. Search the agencies below on X, over the past 7 days, using the patterns in grm-detection.json: current/history handle mentions and replies, plus the configured broad-name searches, all date-filtered.
2. Classify. Keep passenger grievances about a specific service failure. Exclude praise, feature requests and official announcements — the full include/exclude rules are published with the config.
3. Dedupe. One incident = one complaint, however many posts; follow-ups merge into the original.
4. Shared accounts. When agencies share a handle, search it once. Attribute an incident only when the service/operator is identified; if attribution remains unclear, report it once as a shared/unassigned case and never double-count it across agencies.
5. Replies. Detect any visible reply from the agency handle. A reply may be just a forward or a docket number — acknowledged is explicitly not resolved, and each report says which.
6. Measure. Response rate, category counts, approximate response times decoded from post IDs.
7. Publish. Report goes live here, to the archive index, and a summary to Telegram the same morning.
8. Review & improve. After every run the agent reviews what it may have missed; watchlist or rule changes land as a new config version with a changelog entry, and the labeled eval set below is re-checked.

The watchlist

What counts, what doesn't

Known limits

The self-improvement loop

Detection improves through three published artifacts, all in the repo's detection/ folder:

• Config — grm-detection.json: the watchlist, search patterns, include/exclude and reply rules the agent actually executes.
• Changelog — CHANGELOG.md: every config change with the lesson that triggered it.
• Eval set — eval-set.json: 13 labeled complaint posts (5 with known agency replies) plus 3 known-miss drills, seeded from past audits. After any config change, these must still be detected correctly before the change ships.

Detection changelog

Every change to grm-detection.json lands here first: what changed, the lesson that triggered it, and the eval-set outcome after the change.

v1.2 — 2026-10-10 · Chennai handle recovery, broad-name coverage, shared TNSTC/SETC watch

Lesson. Zero-result checks found two bad Chennai watch entries: @drmsrchennai no longer resolves, while @DrmChennai is the active Chennai Division account; @CumtaChennai had no profile/activity evidence, while @CumtaOfficial identifies as CUMTA and is referenced by Chennai One. The same checks surfaced an untagged Gummidipoondi–Chennai Central delay grievance and a Route 105 MTC live-tracking grievance missed by handle-only searches. A broad TNSTC search also found a passenger report of a slow TNSTC trip. The account @tnstc_setc describes itself as TNSTC & SETC, but its official status is not independently linked from a government site; the headlight complaint tagged there does not identify the operator.

Change. Repointed Chennai Division and CUMTA to their currently identified profiles and retained the old entries in handle_history. Replaced inactive TNSTC/SETC handles with the shared candidate @tnstc_setc, preserving the prior handles. Added agency-specific broad search terms and a fourth {search_term} search pattern so untagged service grievances are not dependent on a zero-result week. Added an explicit cross-agency rule: search shared accounts once, assign only when the service/operator is identified, and otherwise count the incident once as shared/unassigned. The joint account’s officiality caveat is explicit in the config.

Detection review. “Live tracking unavailable,” missed stops, pass issuance/refund failures, and slow journeys fit existing categories; no taxonomy or reply-rule change was needed. An official CMRL QR-ticketing notice and its restoration post, suburban derailment news, a general CUMTA criticism, a lane-change question, praise, and posts outside the exact 09:00 IST cutoff were excluded because they were not in-window passenger grievances matching the include rules. Image-led BMTC cases with no visible description are flagged unverified only where BMTC itself acknowledged a complaint. No hourly scheduler data was used.

Eval. Re-checked all 13 labeled complaint cases against the updated watchlist, broad terms, and unchanged inclusion/exclusion rules: every expected agency/category label remains valid. All 3 known-miss drills remain intact: the obsolete Kerala handle returns no results, the MTC legacy handle remains searchable, and praise/feature requests remain excluded. Evaluation set.

Evidence. Chennai Division profile · suburban-delay complaint · Route 105 tracking grievance and reply · CUMTA profile · Chennai One reference · joint TNSTC/SETC profile · TNSTC slow-service complaint · shared headlight complaint.

v1.1 — 2026-10-03 · stale-handle repair

Lesson. The first audits under-counted Chennai and read Kerala as silent. Four of the ten watchlist handles were stale: @mtc_chennai → @MtcChennai, @cmrlchennai → @cmrlofficial, @dtc_india → @dtchq_delhi, and @KSRTC_Kerala was never the Kerala agency at all (it is @keralasrtc). A week of "no complaints found" was really a dead watch.

Change. Corrected all four handles; added handle_history so searches also cover legacy names riders still tag; added the zero_complaint_check rule — a zero week for a known-active agency triggers a handle-liveness check before it can be reported.

Eval. 13 labeled complaint cases + 3 known-miss drills seeded from the 2026-09-26 and 2026-10-03 audits; all cases correctly classifiable under v1.1 rules (legacy-tag thread sibin_91/2105505463530063992 recovered via the reply from the current handle).

v1.0 — 2026-03-30 · initial watchlist

Weekly audit automation created: 10 agencies, 7-day window, complaint/reply detection, "acknowledged is not resolved" reporting rule.