On this page
Keep track of our latest updates and improvements.
Data preview on pipeline editor and k8s spark-operator support with custom FEM images support.
NFS is now available as transfer method for spark-standalone.
New pipeline starters (HTTP, Apify)
Jobs data preview on pipeline editor
Spark-operator support
FEM-Images (custom Spark images)
Pipeline starters
Shared NFS support (for pipeline dependecies) for spark-standalone deployments
Project deletion function
Prevent binary-mode for unsupported entity formats
Dashboard update with correct cache expiration
Disabled pipelines list
Green json creation
Return empty schema for non-explorable repositories
Enhanced spark deployment generation
Database optimizations
Snowflake connector hotfix
Variable definition in the snowflake connector
Fyrefuse now supports the repositories exploration.
Added Snowflake support and enhanced pipelines capabilities.
Snowflake support
API for retrieving explorable repositories
Snowflake support
Added extra repositories parameters to the green json options
Prevent multiple bindings of the same entity for sources and targets
Extract cache and snow parameters from request body
PartitionBy and repartition pipeline options
Added internal prefix to repositories catalogs
New anonymization job available for the Pipelines.
Data explorer query preview and repository exploration (beta).
Anonymization job
Repository exploration
SQL parser on Spark Connect
Data explorer query limit
Data explorer numeric filters
Saving end_date on project save/update
Cache downloaded UDF
Pipelines now support unstructured files in binary mode and Data Explorer supports advanced filtering.
Filters on explorer table
Streaming support for datalayer's tables
Report incremental updates for perpetual streaming pipelines
Binary reading mode for unstructured sources (pdf, images, etc)
Instance list of specific pipeline
Data Explorer join operation
Entities' schema attributes order
Dashboard cache update
Trino datatype mapping
Email templates logo and references
Future days cut in dashboard observation period
Various bugfixes
Terminate_on_complete available for all output-streams
Increased performance of the Data Explorer and improved data quality stats computation and dashboard.
Validation branch to sql jobs
Default values for fem options
Validation rules list on instance details
Size and records on layers/repositories list
Complex types handler to explorer cache writer
Dashboard stats update on certain conditions
Cascade delete pipeline components on pipeline delete
Dashboard struct fields
Major update that includes the first relese of the Datalayer and the brand new dashboard.
The repositories' connectors now have their own maven dependencies list in order to allow spark to automatically download them if needed.
Datalayer (data lakehouse) with the metastore integration
Datalayer tables support inside a Pipeline
Dashboard
Added dependencies list for connectors
spark-sql-kafka dependency version
Save image for a new user
Cached dashboard struct
Refactor of auth middleware
Repositories are now bound to a Fyrefuse project (better project isolation)
Enhanced the green json generation process
Fyrefuse v2.0 is finally out!
Support for technology specific datatypes
Fyrefuse jobs bidirectional communication with Gitlab via Gitlab-API
Data-explorer support for sorting and filtering columns
Websocket for handling data-explorer communication with Trino
Send notification on pipeline events
New engines section
Automatically generate the jdbc connection-string
Notification list API with pagination
Pipeline deployer
Stop instance
Migration from UWSGI to ASGI
SSE Server implementation
Dropped "data-core" concept
Redesigned repositories' Entities schema as json-schema
Repository structure
Dropped "data-core" concept