Project Overview
ATS Relay (formerly CyberScraper) is an independent Python project that collects structured job postings directly from supported employer ATS boards, including listing feeds and posting details. It normalizes source fields while preserving verification evidence for downstream review.
Its acquisition-only interface can collect across role categories within selected supported endpoints, without inheriting security keywords or department filters. Remaining site and geography constraints still apply. Downstream consumers own relevance, ranking, and application decisions. A separate local SQLite-backed Jobs viewer retains technical-role classification, Canadian-market eligibility, and read-only market intelligence.
It is an acquisition and review component within a broader career workflow. Application-lifecycle tracking belongs to the separate career system. Experimental local tracking, synchronization, review CLI, and Fit labels in earlier versions have been retired; old stored data and receipts remain historical evidence.
My responsibility is problem definition, architecture and workflow boundaries, requirements, acceptance criteria, agent direction, generated-output review, and validation. Coding agents, including Codex, performed most source-code implementation. In the live-card example below, I noticed missing location and salary fields, directed the investigation, and checked repaired behavior against employer postings.
Engineering Model: Human Architecture, Agent Implementation
The project used an explicit division of responsibility:
- My engineering ownership: problem framing, architecture, data models, system boundaries, invariants, requirements, acceptance criteria, debugging direction, quality standards, test expectations, and release decisions.
- AI coding-agent implementation: source-code changes, refactors, test implementation, repository audits, implementation proposals, and repetitive code generation under the project constraints.
- Shared iteration loop: the agent proposed or implemented a change; I inspected the result, exercised the real system, identified failures or architectural drift, refined the specification, and accepted or rejected the change based on evidence.
The goal was not to hide AI involvement. The goal was to use AI coding agents as the implementation layer while retaining human ownership of architecture, engineering judgment, and verification.
The Operational Problem
Discovering relevant early-career cybersecurity and technical operations roles through commercial aggregators presents persistent operational challenges:
- Heterogeneous ATS architectures at the original milestone: Employers publish roles across dozens of distinct platforms (Workday, SuccessFactors, Oracle HCM, Taleo, Greenhouse, Lever, Ashby, BambooHR, Dayforce, Jobvite, Rippling, ADP Workforce Now, Phenom, ApplyToJobs, Workable, Pinpoint, and custom static sites). Schemas vary widely. Some expose clean JSON APIs, while others require headless browser DOM traversal.
- Inconsistent & Hallucinated Evidence: Posting dates, salary brackets, remote eligibility, and years of experience (YOE) are frequently omitted or formatted inconsistently. Many commercial platforms hallucinate or assume missing fields.
- Geographic & Seniority Noise: Keyword searches for “security” frequently return senior management, sales engineering, or non-Canadian postings. Personal location preferences can corrupt underlying market data if filtering is applied too early in the ingestion pipeline.
- Historical state requirements: Earlier versions experimented with local applied/dismissed tracking. That responsibility later moved to the broader career workflow; acquisition retains its own source identity and evidence boundaries.
System Architecture
┌─────────────────────────────────────────────────────────────┐
│ Employer Career Endpoints │
│ (Workday · SuccessFactors · Oracle · Greenhouse · Lever) │
└──────────────────────────────┬──────────────────────────────┘
│ HTTP / Playwright (Guarded)
▼
┌─────────────────────────────────────────────────────────────┐
│ Acquisition & Security Layer │
│ • Egress validation • Resource bounds │
│ • Redirect loop guards • Source health telemetry │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Evidence & Normalization Engine │
│ • Canonical URL hashing • Strict field evidence │
│ • Canada remote detection • Canadian municipality dict │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Deterministic Role Classification & Scoring │
│ • Cybersecurity (SOC/DFIR) • IT Support & Operations │
│ • Policy v17 (GIAC, CompTIA, Sysmon, Zeek, KQL, SPL) │
│ • PRAGMA user_version cache invalidation marker │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ SQLite WAL Persistence Layer │
│ • jobs (deduplicated) • job_state (triage history) │
│ • source_health (metrics) • schema_migrations (atomic) │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Desktop Interface & Market Analytics │
│ • Custom SVG dropdown architecture (6 filter controls) │
│ • Official vector flag icons (Canada & US SVG assets) │
│ • Sub-millisecond pre-warmed market queries (< 0.4ms) │
│ • Instant client-side SPA navigation (zero-flicker) │
└─────────────────────────────────────────────────────────────┘
Non-Negotiable Architectural Contracts
- Defensive Data Invariants & Truthful Evidence: Evidence is never fabricated or inferred. Source fields are strictly classified as
present,not_published,unverified, orunavailable. If an employer does not state compensation or experience requirements, the system marks it explicitly asUnavailable. - Raw Evidence Preservation: Original location strings are preserved in
raw_location, while canonical normalizations are stored inlocationusing a dictionary of 60+ Canadian municipal patterns. - Decoupled Classification: Canonical role classification and scoring are completely geography-independent. User search profiles filter only their local queue view, preventing personal preferences from corrupting historical market data.
- Deterministic Identity & SQLite WAL Persistence: Postings are identified by a SHA-256
url_hashcomputed from the stripped canonical URL. Triage state (applied,dismissed) lives in a decoupledjob_statetable. - Cache Invalidation via SQLite
PRAGMA user_version: The scoring policy version is tied directly to the SQLitePRAGMA user_versionheader. When classification regexes change (Policy v17), the background server detects the version bump and safely re-scores stored postings in atomic batches.
The Agentic Engineering Methodology in Practice
I was not acting as a traditional line-by-line programmer on this project. I used AI coding agents as active implementation partners while I retained engineering ownership. The important work was specifying the system precisely enough that an agent could implement it, detecting when the implementation violated the intended behavior, and forcing the architecture back toward a simpler and more reliable design when necessary.
Below are four real incidents that show that engineering loop in practice.
┌──────────────────────────────────────────────────────────────────────────────┐
│ THE AGENTIC ENGINEERING CYCLE │
│ │
│ 1. Precise Architectural Specifications & Non-Negotiable Contracts │
│ │ │
│ ▼ │
│ 2. Coding-Agent Implementation & Repository Work │
│ │ │
│ ▼ │
│ 3. Human Inspection & Edge-Case Discovery │
│ │ │
│ ▼ │
│ 4. Architectural Correction & Anti-Bloat Gatekeeping │
│ │ │
│ ▼ │
│ 5. Test Verification, Acceptance, and Release Decisions │
└──────────────────────────────────────────────────────────────────────────────┘
Case Study 1: Rejecting “Duct-Tape” UI in Favor of Scalable Architecture
- The Problem: When implementing filter dropdowns, the agent defaulted to a patchwork solution: browser-native
<select>pop-ups with emoji flag icons that rendered inconsistently across operating systems and popped up old emoji states on reset. - My Architectural Directive:
“We are not duct-taping this UI together. Build a proper, scalable custom dropdown architecture across all controls. Use official vector SVGs for the US and Canada flags with clean rounded corners, keep the Canadian province dropdown completely free of emojis, and refine cursor states so static containers never trigger text-selection I-beams.”
- The Engineering Outcome: The coding agent implemented a unified, accessible custom dropdown component architecture (
.custom-select-wrap,.custom-select-trigger,.custom-select-menu) backed by official W3C flag geometries, clean provincial typography, and CSSuser-select: none;container guards while keeping hidden native<select>elements synchronized for full automated test compatibility. I evaluated the resulting behavior and accepted the architecture only after it matched the intended interaction model.
Case Study 2: Forensic Quality Control & Data Pipeline Auditing
- The Problem: While all unit tests passed green, manual visual inspection of live job cards revealed that certain Workday postings (e.g. CDW Canada) were missing location and salary metadata pills despite having complete data on the employer’s live portal.
- My Architectural Directive:
“You need to look deeply into this. Trace the entire data pipeline across all ATS modules. Why are location and salary fields dropping out on Workday cards when the data exists in the detail payload?”
- The Engineering Outcome: I directed the investigation and isolated the behavioral gap. The coding agent then implemented facility-code stripping in
normalization.py, added URL slug path extraction as an immediate fallback, and expanded regex extractors inscorer.py(Policy v17) to capture CAD currency variations and hourly compensation. I verified the repaired behavior against live examples rather than treating the passing unit tests as sufficient evidence.
Case Study 3: Relentless Anti-Bloat Gatekeeping
- The Problem: Over weeks of rapid iteration, the application transitioned from an experimental multi-card overview dashboard to a streamlined two-view architecture (Job Board at
/and Market Intelligence at/market). However, the legacy dashboard builder and obsolete templates remained orphaned in the background. - My Architectural Directive:
“We used to have an overview dashboard, but I no longer use it. Run a full-scale dead and stale code audit across the entire repository. Be 100% sure everything can be removed safely before touching anything.”
- The Engineering Outcome: I specified the audit boundary and required proof before deletion. The coding agent performed the AST-based analysis and removed
dashboard.py,dashboard.html,test_dashboard.py, unused module imports, and 258 lines of duplicated CSS, pruning 930+ lines of dead code while preserving a 100% test-suite pass rate.
Case Study 4: Release Criteria & Validation
- The Problem: Preparing the codebase for its official initial milestone release.
- My Architectural Directive:
“Before we cut 1.0.0, run a full pre-version audit across security, contracts, UI themes, and tests. Prune all stray branches and worktrees, verify CI compilation, and ensure the test suite is completely green.”
- The Engineering Outcome: I defined the release gate and acceptance conditions. The coding agent executed the repository-wide checks and implementation work. I reviewed the evidence: zero tracked credentials, verified light and dark theme behavior, clean Git state, and a fully passing 496 / 496 automated test suite. I accepted the
v1.0.0milestone on that evidence. Passing checks supported that release decision; they did not prove the absence of defects or complete source coverage.
Verification & Operational Metrics
- Unit Test Suite: 496 passing automated tests covering all 48 test modules (
Ran 496 tests in 24.6s. OK (0 failures)). - Target Coverage: 150+ monitored employer targets across 26 dedicated ATS adapter modules (
workday,greenhouse,ashby,lever,dayforce,oracle_hcm,successfactors,workable,pinpoint,rippling,bamboohr, etc.). - Ingestion & Throughput: Benchmark-audited ingestion rate of 7,877.7 rows/sec and concurrent target scheduling of 1,090 targets/sec at concurrency=20.
- Page Load Latency: Pre-warmed market intelligence queries execute in < 0.4ms.
- Scoring Engine: Monotonically versioned rescoring engine (
Policy v17) mapping SOC, DFIR, SIEM (KQL/SPL), and industry certifications (GCIH,GSEC,Security+,CySA+). - Release Milestone: Formally tagged
v1.0.0release.
Later Delivery Work: Acquisition Boundaries and Partial Results
The v1.0.0 metrics above describe the original milestone. Later work, merged on October 3, 2026, strengthened the boundary between collecting employer evidence and making career-workflow decisions.
Pinned acquisition inputs
The acquisition CLI gained a reviewed run manifest that pins the code revision, dependency lock, and selected source/profile scope. Readiness checks block acquisition when required inputs, export permissions, transport, or browser capability are unavailable. The launchers reuse the project environment and the existing export contract.
This gives an agent a bounded collection task with identifiable inputs. The collector returns source observations; ranking, application tracking, and consequential employer-facing actions remain separate concerns. A boundary-review receipt records an operator’s review, but does not create or prove operating-system isolation.
Retain valid postings without hiding a failed read
A Workday requisition with a blank or placeholder description previously rejected successful sibling postings. The correction retains validated postings, counts unusable requisitions, and reports partial, unverified coverage after the listing pages complete.
The acquisition export has no lifecycle authority. In the desktop pipeline, returned valid postings may refresh while Source Health remains in error and missing jobs are not aged or closed on the strength of that partial read. If every description is unusable, the result remains an error. Network, malformed-payload, and identity failures still reject the batch.
Regression fixtures cover mixed and all-bad descriptions, failures across listing pages, fatal detail failures, desktop detail selection, and duplicate observations. Canonical counting preserves the observations while avoiding an inflated unique-coverage claim. Linux and Windows CI passed for both merged changes.
This extends the project’s verification method: useful data can survive a bounded failure, but the failure remains visible and cannot authorize conclusions about absent postings. Acquisition volume is not a count of unique eligible jobs, applications, or placements.
Key Takeaways: Engineering with AI Coding Agents
ATS Relay reinforced three principles for me:
- Specifications are an engineering artifact: When AI agents can implement code quickly, architecture, contracts, invariants, acceptance criteria, and system boundaries become even more important. The quality of the implementation depends heavily on the quality of those constraints.
- Human engineering ownership still matters: Coding agents can implement quickly, but they can also introduce technical debt, local fixes, and architectural drift. I treated the agent as an implementation partner, not as the source of product judgment.
- Verification cannot be delegated blindly: Automated tests are necessary, but they are not sufficient. Several important failures only appeared through manual inspection of the live system. My role was to identify those mismatches, direct root-cause work, and require evidence before accepting a fix.
The demonstrated contribution is requirements, architecture, agent direction, output review, and validation. Coding agents did most implementation. The missing-metadata repair illustrates why I check source evidence and actual behavior alongside automated tests; it does not establish measured time savings, complete acquisition coverage, or a hiring outcome.