v0.8.0
This release brings configuration management into Flowthru's type-safe ecosystem, and adds HTTP data pulling with optional storage-based caching.
What's New
- Configuration as Typed Catalog: Configuration sections are now bound as strongly-typed catalog items, so steps can depend on configuration the same way they depend on data. Use the
[FlowthruConfig]attribute and[ConfigSection]properties to wire configuration sections (fromappsettings.jsonor environment variables) directly into your step parameters. This eliminates string-based configuration lookups and moves configuration errors to compile-time.- Idiomatic .NET Integration: Flowthru now uses the standard
Microsoft.Extensions.Configurationsystem rather than custom configuration plumbing. Extensions are configured through the standard DI container, and all examples have been migrated. See KedroSpaceflights.Custom for a complete example of configuration as catalog in action.
- Idiomatic .NET Integration: Flowthru now uses the standard
- HTTP Provider with Storage Caching: You can now pull data over HTTP using the
HttpStorageMediumProviderfrom the new HTTP extension. Optionally cache downloaded files locally to avoid repeated HTTP calls; specify a cache directory and maximum age, and cached files are automatically validated and refreshed. RetailDataSplitFlow demonstrates HTTP pulling with caching. - Enhanced Metadata Surface: The metadata provider API now has better visibility into flow exports and improved default provider behavior, making it easier to inspect and document your pipelines after execution.
🚀 Features
- pull over HTTP, storage cache (8d430203)
- better metadata surface and default provider behavior (d9df524f)
- migrate to dotnet IConfiguration (d3c8e50d)
- extensions now use idiomatic C# config system (77100c0c)
- configuration as catalog (ed8e3cb4)
🩹 Fixes
- resolve issue with GitHub pre-releases (95b15373)
- resolve CI orphaned head issue (ec5c7237)
- resolve issues with manual dispatch deployments hitting NX versioning clothesline. (07c1da41)
❤️ Thank You
- Spencer Elkington