Skip to content

fastflowtransform.config.packages

PackageSpec

Bases: BaseModel

One entry from packages.yml, for example:

packages: - name: fft_utils path: "../fft_utils" models_dir: "models"

Or (shorthand mapping form):

fft_utils: "../fft_utils"

For git-based packages:

  • name: shared_package_git git: "https://github.com/org/repo.git" subdir: "path/inside/repo" # one of the revision selectors below is optional: # - ref: "main" (generic alias, mapped to rev) # - rev: "abc1234" (commit SHA) # - tag: "v1.2.3" # - branch: "main" # models_dir: "models" # optional, default "models"
Source code in src/fastflowtransform/config/packages.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
class PackageSpec(BaseModel):
    """
    One entry from packages.yml, for example:

      packages:
        - name: fft_utils
          path: "../fft_utils"
          models_dir: "models"

    Or (shorthand mapping form):

      fft_utils: "../fft_utils"

    For git-based packages:

      - name: shared_package_git
        git: "https://github.com/org/repo.git"
        subdir: "path/inside/repo"
        # one of the revision selectors below is optional:
        #   - ref: "main"      (generic alias, mapped to `rev`)
        #   - rev: "abc1234"   (commit SHA)
        #   - tag: "v1.2.3"
        #   - branch: "main"
        # models_dir: "models"  # optional, default "models"
    """

    model_config = ConfigDict(extra="forbid")

    name: str

    # Exactly one of `path` or `git` must be set.
    path: str | None = None
    git: str | None = None

    # Optional git parameters (ignored for path-based packages).
    #
    # "ref" is a user-facing alias (branch/tag/commit); internally we map it to `rev`
    # if no more-specific selector (rev/tag/branch) is provided.
    ref: str | None = None
    rev: str | None = None
    tag: str | None = None
    branch: str | None = None
    subdir: str | None = None

    # Where models live inside the package root (default: "models").
    # This can be overridden by the package's own project.yml (models_dir),
    # but packages.yml always wins if set explicitly.
    models_dir: str = "models"

    # Optional constraint for the package's manifest version (semver expression).
    # Example: ">=1.0.0,<2.0.0"
    version: str | None = None

    @model_validator(mode="after")
    def _validate_source(self) -> PackageSpec:
        """
        Ensure that exactly one of `path` or `git` is set.
        Also treat `ref` as a generic alias for `rev` when no other
        more-specific selector (rev/tag/branch) is given.
        """
        has_path = bool(self.path)
        has_git = bool(self.git)

        if has_path == has_git:
            # Either both set or both unset → error.
            raise ValueError(
                f"Package '{self.name}': exactly one of 'path' or 'git' must be set "
                "in packages.yml."
            )

        # If user provided a generic `ref` but no explicit rev/tag/branch,
        # map it to `rev` so downstream resolver can just look at rev/tag/branch.
        if self.ref and not (self.rev or self.tag or self.branch):
            self.rev = self.ref

        return self

PackagesConfig

Bases: BaseModel

Top-level representation of packages.yml.

We accept two shapes:

1) Explicit:

 packages:
   - name: fft_utils
     path: "../fft_utils"
     models_dir: "models"

2) Shorthand mapping:

 fft_utils: "../fft_utils"
 other_pkg:
   path: "../other"
   models_dir: "dbt_models"
Source code in src/fastflowtransform/config/packages.py 
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
class PackagesConfig(BaseModel):
    """
    Top-level representation of packages.yml.

    We accept two shapes:

      1) Explicit:

         packages:
           - name: fft_utils
             path: "../fft_utils"
             models_dir: "models"

      2) Shorthand mapping:

         fft_utils: "../fft_utils"
         other_pkg:
           path: "../other"
           models_dir: "dbt_models"
    """

    model_config = ConfigDict(extra="forbid")

    packages: list[PackageSpec] = Field(default_factory=list)

load_packages_config 

load_packages_config(project_dir)

Read packages.yml under project_dir and return a strict PackagesConfig.

If the file does not exist, we return an empty config (no packages).

Source code in src/fastflowtransform/config/packages.py 
148
149
150
151
152
153
154
155
156
157
158
159
160
def load_packages_config(project_dir: Path) -> PackagesConfig:
    """
    Read packages.yml under `project_dir` and return a strict PackagesConfig.

    If the file does not exist, we return an empty config (no packages).
    """
    cfg_path = project_dir / "packages.yml"
    if not cfg_path.exists():
        return PackagesConfig()

    raw = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {}
    norm = _normalize_raw_packages(raw)
    return PackagesConfig.model_validate(norm)