Layer separation¶
The codebase is organised into 9 logical layers (per the /understand knowledge-graph analysis). Every file belongs to exactly one. Imports flow downward — a higher layer may import from a lower one, never the reverse.
flowchart TB
L1["1. Public API
src/index.ts · Definitions · Transactions"]
L2["2. Pipeline orchestration
Builder · Executor · phase classes"]
L3["3. Pipeline mediator
Browser · Form · Elements · Dashboard · Login · OTP · BalanceResolve · …"]
L4["4. Pipeline strategy + registry
Fetch + Scrape strategies · per-bank PipelineDescriptor · WK registry"]
L5["5. Pipeline types
BasePhase · Procedure · Option · PiiRedactor · PipelineContext"]
L6["6. Legacy scrapers (deprecated)
BaseScraper · BaseScraperWithBrowser · per-bank legacy classes"]
L7["7. Common utilities (deprecated)
src/Common/ helpers consumed by both generations"]
L8["8. Tests
src/Tests/ — Unit / Pipeline / E2eMocked / E2eReal / Helpers"]
L9["9. Build & CI
GitHub Actions · husky · Dockerfile.ci-mirror · tsup · biome · eslint"]
L1 --> L2
L1 --> L6
L2 --> L3 --> L4 --> L5
L6 --> L7
L6 --> L5
L8 --> L1
L8 --> L2
L8 --> L3
L9 -.->|"orchestrates"| L8
Layer details¶
| # | Layer | Path | Status |
|---|---|---|---|
| 1 | Public API | src/index.ts, src/Definitions.ts, src/Transactions.ts, src/Scrapers/Registry/Factory.ts | Stable |
| 2 | Pipeline orchestration | src/Scrapers/Pipeline/Core/, src/Scrapers/Pipeline/Phases/, src/Scrapers/Pipeline/Interceptors/ | Canonical |
| 3 | Pipeline mediator | src/Scrapers/Pipeline/Mediator/ (BalanceResolve, Browser, Dashboard, Elements, Form, Login, Network, OtpFill, OtpTrigger, Scrape, …) | Canonical |
| 4 | Pipeline strategy + registry | src/Scrapers/Pipeline/Strategy/, src/Scrapers/Pipeline/Banks/, src/Scrapers/Pipeline/Registry/, src/Scrapers/Registry/Factory.ts | Canonical |
| 5 | Pipeline types | src/Scrapers/Pipeline/Types/, src/Scrapers/Base/{Interface, ErrorTypes, ScraperError, Config/, Interfaces/} | Shared infra |
| 6 | Legacy scrapers | src/Scrapers/Base/{BaseScraper, BaseScraperWithBrowser, BaseScraperHelpers, ConcreteGenericScraper, GenericBankScraper}.ts, src/Scrapers/{Behatsdaa, BeyahadBishvilha, Mizrahi}/, src/Scrapers/Registry/ScraperRegistryLeumiToYahav.ts (Mizrahi only) | Deprecated |
| 7 | Common utilities | src/Common/ (Browser, CamoufoxLauncher, Fetch, Navigation, OtpDetector, OtpHandler, ResultFormatter, SafeScreenshot, SelectorResolver, Storage, Waiting, …) | Mostly deprecated — pipeline uses only Debug.ts |
| 8 | Tests | src/Tests/Unit/, src/Tests/E2eMocked/, src/Tests/E2eReal/, src/Tests/Helpers/, src/Tests/Tools/ | Always-on gate |
| 9 | Build & CI | .github/workflows/, .husky/, tsup.config.ts, biome.json, eslint.config.mjs, tsconfig.json | Always-on gate |
What lives in layer 5 vs layer 6¶
Layer 5 (shared infra — used by Pipeline 107 times) keeps shipping unchanged:
src/Scrapers/Base/Interface.ts—IScraper,IScraperScrapingResult,ScraperOptions,ScraperCredentialssrc/Scrapers/Base/ErrorTypes.ts—ScraperErrorTypesenumsrc/Scrapers/Base/ScraperError.ts— error classsrc/Scrapers/Base/Config/LoginConfig*.ts— declarative login config typessrc/Scrapers/Base/Interfaces/**— all sub-interfaces
Layer 6 (legacy) still works through createScraper but is on the migration path:
- The 3 legacy bank dirs (Behatsdaa, BeyahadBishvilha, Mizrahi) — Leumi + Yahav migrated to Pipeline
- The 5 legacy base classes (
BaseScraper,BaseScraperWithBrowser,BaseScraperHelpers,ConcreteGenericScraper,GenericBankScraper) - The legacy registry (
ScraperRegistryLeumiToYahav.ts)
Import direction enforcement¶
The architecture validator (lint:architecture) runs on every commit and rejects:
- Imports from
src/Scrapers/Pipeline/intosrc/Common/orsrc/Scrapers/Base/{BaseScraper*, ConcreteGenericScraper} - Imports from layer 1 (Public API) into any legacy implementation
- Cross-mediator imports within layer 3 (BALANCE-RESOLVE cannot reach into SCRAPE state directly)
Source: src/Tests/Tools/lint-and-validate.ts + the 3 architecture canaries under src/Scrapers/Pipeline/EslintCanaries/.