How it Works
Pipeline overview
Location → parse → RawDataset → INSERT into SQLite → Repository/Services → Command output
Each command calls build_database() which:
-
Parses all sources into the in-memory SQLite database (two-phase traversal, see below)
-
Applies filters (
DatabaseFilterProcessor) — removes out-of-scope requirements and applies user-definedfilters:blocks -
Runs lifecycle validation (warns on DEPRECATED/OBSOLETE references)
-
Commands then query via
RequirementsRepositoryand service layer
Two-phase graph traversal
The requirement graph is a directed graph of URNs connected by two edge types:
-
import— "I reference requirements from this URN" (upward, toward requirement definitions) -
implementation— "this URN provides evidence for my requirements" (downward, toward code/tests)
CombinedRawDatasetsGenerator traverses this graph in two phases:
Phase 1 — import chain (recursive DFS, full insert)
Follows imports: sections recursively, depth-first. For each node, all five data types are fully inserted into SQLite: requirements, SVCs, MVRs, annotations, and test results. Cycle detection raises CircularImportError.
Phase 2 — implementation chain (recursive, metadata-only insert)
Follows implementations: sections recursively. Think library-uses-library — lib-a can itself have implementations (lib-b → lib-c), all of which may contribute test evidence for the initial URN’s requirements.
For each implementation node, requirements.yml is parsed (validation runs) but only the URN metadata is inserted — the requirement rows are excluded. All other files (SVCs, MVRs, annotations, test results) are inserted normally; SQLite FK constraints automatically discard rows that reference requirements outside the current scope. Cycle detection raises CircularImplementationError.
imports: sections of implementation nodes are not followed — an implementation’s own imports belong to a different scope.
|